key: cord-0992703-ttlbbspm
authors: Freire-Paspuel, Byron; Garcia-Bereguiain, Miguel Angel
title: Clinical Performance and Analytical Sensitivity of Three SARS-CoV-2 Nucleic Acid Diagnostic Tests
date: 2021-02-26
journal: Am J Trop Med Hyg
DOI: 10.4269/ajtmh.20-1484
sha: 0f8f7f05fb9fad5a9b2ae5171833008a068e54ea
doc_id: 992703
cord_uid: ttlbbspm

Hundreds of RT-qPCR kits are available in the market for SARS-CoV-2 diagnosis, some of them with emergency use authorization (EUA) by the Food Drug Administration (FDA) or their country of origin agency, but also many of them without any independent clinical performance evaluation. We performed a clinical evaluation for two Chinese SARS-CoV-2 RT-PCR kits available in South America, COVID-19 Nucleic Acid Test Kit (eDiagnosis Biomedicine, Wuhan, China) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech, Changsha, China), for RT-qPCR SARS-CoV-2 diagnosis using the FDA EUA 2019-nCoV CDC kit (IDT, Coralville, IA) as gold standard. We found an excellent clinical performance and analytical sensitivity for both kits with sensitivity values of 100% and 95.3% and estimated limit of detection of 500 copies/mL and 1,000 copies/mL, for eDiagnosis and Sansure Biotech kits, respectively. COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) are both made in China and hold EUA by the Chinese CDC. Also, Sansure Biotech kit has EUA by the FDA. In conclusion, our results endorse the use of these two commercially available kits imported to Ecuador for SARS-CoV-2 diagnosis, as they had the similar clinical performance as the gold standard from the CDC.

The COVID-19 pandemic has challenged public health systems worldwide, from patient care to epidemic management, but also to even guarantee the quality of SARS-CoV-2-related diagnostic products. 1 Hundreds of in vitro SARS-CoV-2 RT-qPCR diagnostic kits are available on the market, some of them have received emergency use authorization (EUA) from the U.S. Food & Drug Administration (FDA) or other international agencies like Chinese CDC (C-CDC), whereas many others only report clinical validations made by manufacturers. The U.S. CDC (CDC) designed the FDA EUA 2019-nCoV CDC kit (IDT), based on N1 and N2 gene targets to detect SARS-CoV-2 and RNase P as an RNA extraction quality control that is considered a gold standard worldwide for SARS-CoV-2 RT-PCR diagnosis. [2] [3] [4] [5] Among the commercial kits available in South America for SARS-CoV-2 diagnosis, several are made in China. For instance, COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) are both multiplex SARS-CoV-2 RT-qPCR kit for N and ORF1ab viral gene targets and RNase P as an RNA extraction quality control. Although both of them are authorized for clinical use in China, only the Sansure Biotech kit holds the FDA EUA. 6 We also have previously reported that other made in China SARS-CoV-2 RT-PCR kits lacking FDA EUA but holding C-CDC EUA had a great clinical performance and analytical sensitivity. 7 We herein present a comparison of the analytical and clinical performance of COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) for SARS-CoV-2 RT-qPCR diagnosis from nasopharyngeal samples, using the CDC protocol as a gold standard.

Study design. Ninety seven clinical specimens (nasopharyngeal swabs collected on 0.5 mL TE pH 8 buffer) were included on this study. Also, negative controls (TE pH 8 buffer) were included as control for carryover contamination, one for each set of RNA extractions.

RNA extraction and RT-qPCR for SARS-CoV-2 diagnosis using 2019-nCoV CDC kit. All the samples included on the study were tested following an adapted version of the CDC protocol: (1) using AccuPrep Viral RNA extraction kit (Bioneer, Daejeon, South Korea) as an alternate RNA extraction method, and (2) using CFX96 BioRad instrument. [8] [9] [10] [11] [12] RT-qPCR for SARS-CoV-2 diagnosis using COVID-19 Nucleic Acid Test Kit (eDiagnosis). Same RNA extractions from all the samples included in the study were tested using COVID-19 Nucleic Acid Test Kit (eDiagnosis), following the manufacturer's manual. To avoid RNA degradation, samples were processed for 2019-nCoV CDC kit and COVID-19 Nucleic Acid Test Kit (eDiagnosis) within the same day, or frozen at −80 C to be processed the next day.

RT-qPCR for SARS-CoV-2 diagnosis using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech). Seventy four from the total 97 RNA extractions included in the study were tested using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech). Although this kit includes a lysis buffer for an alternative RNA isolation without using RNA extraction kits, we did not follow that protocol. Also, although the manufacturer's manual recommends a final volume reaction of 50 μL (30 μl of PCR Master Mix + 20 μL of RNA extraction), we used a final volume reaction of 15 μL (9 μL of PCR Master Mix + 6 μL of RNA extraction), keeping the proportion of reagents recommended by the manufacturer. To avoid RNA degradation, samples were processed for 2019-nCoV CDC kit and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) within the same day, or frozen at −80 C to be processed the next day.

Analytical sensitivity. Limit of detection (LoD) was performed using the commercially available 2019-nCoV N positive control (IDT); provided at 200.000 genome equivalents/ mL, it was used for calibration curves to obtain the viral loads of the samples. Viral loads can be expressed as copies/μL of RNA extraction or copies/mL of sample; the conversion factor is 200, as 0.2 mL of sample is used for RNA extraction and 40 μL is used as final elution volume of RNA extraction.

Statistics. 95% confident intervals were calculated using Jamovi statistical software.

Ethics statement. All samples have been submitted for routine patient care and diagnostics. Ethics approval was not sought because the study involves laboratory validation of test methods and the secondary use of anonymous pathological specimens that falls under the category "exempted" by Comité de Etica para Investigación en Seres Humanos from "Universidad de Las Américas."

Clinical performance and estimation of LoD of COVID-19 Nucleic Acid Tests Kit (eDiagnosis) using 2019-nCoV CDC kit as a gold standard. Ninety seven samples were tested for SARS-CoV-2 following both COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV CDC kit protocols as described on the methods. For the 2019-nCoV CDC EUA kit, 63 samples tested positive and 34 samples tested negative (Table 1 and Supplemental Tables 1 and 2 ). Thirty-two of 34 samples tested negative for 2019-nCoV CDC kit were also SARS-CoV-2 negative for COVID-19 Nucleic Acid Test Kit (eDiagnosis), so the specificity obtained in our study was 94.1% (91.0-97.2, CI 95%). The two "false-positive" samples had Ct values > 38 (Supplemental Tables 1 and 2) that are associated to low viral loads. As the LoD of COVID-19 Nucleic Acid Test Kit (eDiagnosis) is 2.5 copies/μL according to the manufacturers, these two "false-positive" samples would be actually true-positive SARS-CoV-2 samples that are below the LoD of the CDC protocol. [8] [9] [10] [11] [12] So the real specificity for COVID-19 Nucleic Acid Test Kit (eDiagnosis) is 100%.

For the 63 SARS-CoV-2-positive samples for 2019-nCoV CDC EUA kit, all were also positive for COVID-19 Nucleic Acid Test Kit (eDiagnosis), resulting in a sensitivity of 100% (Table 1  and Supplemental Tables 1 and 2) .

As the LoD is defined as the lowest viral load in which all replicates are detected (100% sensitivity), our data indicate that the LoD for COVID-19 Nucleic Acid Test Kit (eDiagnosis) would be around 2.5 copies/μL of RNA extraction (500 viral RNA copies/ mL of sample), as no failure to detect SARS-CoV-2-positive samples above this threshold was found (Supplemental Table 1) .

Clinical performance and estimation of LoD of 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) using 2019-nCoV CDC kit as a gold standard. Seventy-four samples were tested for SARS-CoV-2 using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) and 2019-nCoV CDC kit protocols as described on the methods. For the 2019-nCoV CDC EUA kit, 43 samples tested positive and 31 samples tested negative (Table 1 and Supplemental Table 1 ). Twentyeight of 31 samples tested negative for 2019-nCoV CDC kit were also SARS-CoV-2 negative using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech), so the specificity obtained in our study was 90.3% (87.3-93.7, CI 95%). The three "false-positive" samples had Ct values > 37 (Supplemental Table 1 ) that are associated to low viral loads. As the LoD of 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) is 1 copie/μL according to the manufacturers, these three "false-positive" samples would be actually true-positive SARS-CoV-2 samples that are below the LoD of the CDC protocol. [7] [8] [9] [10] [11] So the real specificity for 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) is 100%.

For the 43 SARS-CoV-2-positive samples for 2019-nCoV CDC EUA kit, 41 samples were also positive for 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech), resulting in a sensitivity of 95.3% (93.1-97.5, CI 95%; Table 1 and Supplemental Table 1 ).

As the LoD is defined as the lowest viral load in which all replicates are detected (100% sensitivity), our data indicate that the LoD for 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) would be above five copies/μL of RNA extraction (1,000 viral RNA copies/mL of sample), as only one failure (40/41; 97.6% sensitivity) to detect SARS-CoV-2-positive samples above this threshold was found (Supplemental Table 1 ).

Although the main limitation of our study is the sample size (97 specimens), our results support that both COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) had a great clinical performance compared with 2019-nCoV CDC EUA, with sensitivity values up to 100% and 95.3%, and specificity values of 100%.

As we have described on the Results, we could estimate the LoD of COVID-19 Nucleic Acid Tests Kit (eDiagnosis) on approximately 500 viral RNA copies/mL of sample which is on agreement with the LoD detailed at the manufacturer's manual. Moreover, we could estimate the LoD of 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) to be around on 1,000 viral RNA copies/mL of sample, slightly over the LoD of 200 copies/mL detailed at the manufacturer's manual. However, the LoDs obtained for both kits are quite acceptable for a reliable SARS-CoV-2 diagnosis considering the viral load frequency population distributions. 13, 14 On Table 2 , analytical parameters and other characteristics for COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) are summarized. As we have detailed on the Introduction, only the Sansure Biotech kit has both FDA EUA and C-CDC EUA, whereas the eDiagnosis only holds C-CDC EUA. Moreover, we have already published another report showing a great clinical performance from another C-CDC EUA kit, from Da an Gene company. 7 In addition, we have also already published other clinical evaluation reports for Spanish, Canadian, and 

South Korean RT-PCR kits with really variable results in terms of sensitivity and LoD. 7, 8, [15] [16] [17] Although further studies would be necessary involving many more SARS-CoV-2 RT-PCR diagnostic kits, we already found an interesting trend: only those kits with their country of origin clinical use authorization had an acceptable clinical performance and analytical sensitivity.

Considering the worldwide high demand of reagents for SARS-CoV RT-qPCR diagnosis, independent clinical performance and analytical sensitivity evaluations are necessary to guarantee the quality of the supplies in the market for every country in the world, particularly for developing countries usually lacking of reliable regulatory agencies. We claim for the role of universities at developing countries like Ecuador, not only to assess quality evaluations like the ones described on this work but also to support an extremely needed massive SARS-CoV-2 diagnosis. N, ORF1ab 1,000 viral copies/mL (200 viral copies/mL) FDA C-CDC C-CDC = Chinese CDC; EUA = Emergency Use Authorization; FDA = Federal Drug Administration; LoD = limit of detection.

Clinical, molecular, and epidemiological characterization of the SARS-CoV-2 virus and the coronavirus disease 2019 (COVID-19), a comprehensive literature review

US CDC real-Time reverse Transcription PCR panel for detection of severe acute respiratory syndrome coronavirus 2

Interim Guidelines for Collecting, Handling, and Testing Clinical Specimens from Persons for Coronavirus Disease 2019 (COVID-19)

Comparison of abbott ID now, diasorin simplexa, and CDC FDA EUA methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from individuals diagnosed with COVID-19

Comparative performance of SARS-CoV-2 detection Assays using seven different primer/probe sets and one assay kit

Analytical and clinical evaluation of two RT-qPCR SARS-CoV-2 diagnostic tests with emergency use authorization in Ecuador

Evaluation of nCoV-QS (MiCo BioMed) for RT-qPCR detection of SARS-CoV-2 from nasopharyngeal samples using CDC FDA EUA qPCR kit as a gold standard: an example of the need of validation studies

Cottontipped plastic swabs for SARS-CoV-2 RT-qPCR diagnosis to prevent supply shortages

Sample pooling of RNA extracts to speed up SARS-CoV-2 diagnosis using CDC FDA EUA RT-qPCR kit

One health" inspired SARS-CoV-2 surveillance: the galapagos Islands experience

Analytical sensitivity and clinical performance of a triplex RT-qPCR assay using CDC N1, N2 and RP targets for SARS-CoV-2 diagnosis

Suppression of a SARS-CoV-2 outbreak in the Italian municipality of Vo'

SARS-CoV-2 viral load Assessment in respiratory samples

Poor sensitivity of AccuPower SARS-CoV-2 real Time RT-PCR kit (Bioneer, South Korea)

Analytical and clinical comparison of Viasure (CerTest Biotec) and 2019-nCoV CDC (IDT) RT-qPCR kits for SARS-CoV2 diagnosis

Low clinical performance of "Isopollo COVID19 detection kit" (Monitor, South Korea) for RT-LAMP based SARS-CoV-2 diagnosis: a call for action against low quality products for developing countries

Acknowledgments: We thank Tannya Lozada from "Dirección General de Investigación de la Universidad de Las Américas," and also the authorities from Universidad de Las Américas, for their logistic support to make SARS-CoV-2 diagnosis possible. The American Society of Tropical Medicine and Hygiene has waived the Open Access fee for this article due to the ongoing COVID-19 pandemic.Financial support: This study was funded by Universidad de Las Americas (Quito, Ecuador).Disclosure: All authors contributed to study conceptualization, experimental procedures, and revision and approval of the final version of the manuscript. Byron Freire-Paspuel and Miguel Angel García Bereguiain analyzed the data and wrote the manuscript.

Authors' addresses: Byron Freire-Paspuel and Miguel Angel Garcia-Bereguiain, One Health Research Group, Universidad de Las Americas, Quito, Ecuador, E-mails: byron.freire.paspuel@udla.edu.ec and magbereguiain@gmail.com.This is an open-access article distributed under the terms of the Creative Commons Attribution (CC-BY) License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.