key: cord-1042868-jqv02a80
authors: Nalla, Arun K.; Casto, Amanda M.; Huang, Meei-Li W.; Perchetti, Garrett A.; Sampoleo, Reigran; Shrestha, Lasata; Wei, Yulun; Zhu, Haiying; Jerome, Keith R.; Greninger, Alexander L.
title: Comparative Performance of SARS-CoV-2 Detection Assays Using Seven Different Primer-Probe Sets and One Assay Kit
date: 2020-05-26
journal: J Clin Microbiol
DOI: 10.1128/jcm.00557-20
sha: f41ecc7bc72c686368af60ffaf39d070e676923c
doc_id: 1042868
cord_uid: jqv02a80

Nearly 400,000 people worldwide are known to have been infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) beginning in December 2019. The virus has now spread to over 168 countries including the United States, where the first cluster of cases was observed in the Seattle metropolitan area in Washington. Given the rapid increase in the number of cases in many localities, the availability of accurate, high-throughput SARS-CoV-2 testing is vital to efforts to manage the current public health crisis. In the course of optimizing SARS-CoV-2 testing performed by the University of Washington Clinical Virology Lab (UW Virology Lab), we evaluated assays using seven different primer-probe sets and one assay kit. We found that the most sensitive assays were those that used the E-gene primer-probe set described by Corman et al. (V. M. Corman, O. Landt, M. Kaiser, R. Molenkamp, et al., Euro Surveill 25:2000045, 2020, https://doi.org/10.2807/1560-7917.ES.2020.25.3.2000045) and the N2 set developed by the CDC (Division of Viral Diseases, Centers for Disease Control and Prevention, 2020, https://www.cdc.gov/coronavirus/2019-ncov/downloads/rt-pcr-panel-primer-probes.pdf). All assays tested were found to be highly specific for SARS-CoV-2, with no cross-reactivity with other respiratory viruses observed in our analyses regardless of the primer-probe set or kit used. These results will provide valuable information to other clinical laboratories who are actively developing SARS-CoV-2 testing protocols at a time when increased testing capacity is urgently needed worldwide.

I n late December 2019, a cluster of cases of pneumonia of unclear etiology was first noted in Wuhan City in the Hubei Province of China (1) . The etiology of these pneumonia cases, a novel type of coronavirus, was identified on 7 January 2020 (1). This novel coronavirus has now been named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) while the disease it causes is known as coronavirus disease 2019 (COVID-19) (2) .

Though the epidemic was originally concentrated in China, it has now spread around the world with approximately 380,000 known cases (see Note S1 in the supplemental material) as of 23 March 2020 (3) . Cases outside China were observed early in the epidemic with the first detected in Thailand on 13 January 2020 (4) . Soon afterward, cases were also identified in other East Asian countries including Japan and South Korea (1) . The virus then spread to Europe and the Middle East, with the nations of Italy and Iran having particularly large numbers of identified cases per capita (3) . Cases have now been identified, and deaths due to COVID-19 reported, on all 6 populated continents (3) .

The Centers for Disease Control and Prevention (CDC) confirmed the first case of COVID-19 in the United States on 21 January. The infected person was a 35-year-old man who had recently returned to his home in Snohomish County, WA, after travelling to Wuhan City (5) . No additional cases of COVID-19 were identified in Washington State until 28 February when two new cases were confirmed, one in Snohomish County and one in neighboring King County, where Seattle is located (6, 7) . Since 28 February, the number of cases of COVID-19 in Washington has steadily increased and currently stands at 2,096 (3). In response to the rapidly increasing number of confirmed and suspected cases of COVID-19 in the Seattle metropolitan area, the Clinical Virology Laboratory at the University of Washington (UW Virology Lab) began testing clinical specimens for SARS-CoV-2. Prior to and since making this testing service available, we have endeavored both to optimize the performance of our assay and to increase the rate at which we are able to test samples. We report here our observations comparing three different RNA extraction methods. We also compare the performances of SARS-CoV-2 detection assays using seven different primer-probe sets and one assay kit.

Samples. Three sets of samples were used in our analyses. First, we used a set of approximately 300 clinical respiratory samples sent to the UW Virology Lab for respiratory virus testing. These samples were in the form of nasopharyngeal or oropharyngeal swabs in viral transport medium and had not previously been tested for SARS-CoV-2. Second, we used a collection of nasal swabs in viral transport medium that are used to validate all assays performed by the UW Virology Lab. This collection includes samples SARS-CoV-2 detection assays. We used a total of 7 different primer-probe sets in our SARS-CoV-2 detection assays (see Table S1 in the supplemental material). The University of Washington (UW) RdRp primer-probe set was designed by the UW Virology Lab. Three additional primer-probe sets were designed as described in the work of Corman et al. (8) ; these will be referred to as the Corman N-gene, RdRp, and E-gene primer-probe sets. The Centers for Disease Control and Prevention (CDC) N1, N2, and N3 sets were developed by the CDC and have been published on the CDC website (9) .

For all of the above primer-probe sets, real-time RT-PCR assays were performed using the AgPath-ID One Step RT-PCR kit (Life Technologies). Twenty-five microliters of reaction mix consists of 2ϫ RT-PCR buffer, 25ϫ enzyme mix, primers-probes, and 5 l of extracted nucleic acid. Primer-probe concentrations were as recommended in the work of Corman et al. (8) and by a CDC-recommended protocol (9) . RT-PCR was performed on an ABI 7500 real-time PCR system (Applied Biosystems) with the following cycle parameters: 10 min at 48°C for reverse transcription and 10 min of inactivation at 95°C followed by 40 cycles of 15 s at 95°C and 45 s at 60°C.

We also tested samples for SARS-CoV-2 using the BGI RT-PCR detection kit (BGI). These assays were conducted according to the kit manufacturer's instructions.

A negative (human specimen control) was included in every RNA extraction procedure, and a nontemplate (water) control was included in every RT-PCR run. An internal control amplification, either RNase P or EXO (10) , was performed to monitor RNA extraction and RT-PCR quality.

Determination of SARS-CoV-2 sample copy number. Dilutions of one SARS-CoV-2-positive clinical sample (SC5688) were used to evaluate the sensitivities of SARS-CoV-2 detection assays. To determine the viral copy number in SC5688, we obtained SARS-CoV-2 whole genomic RNA from the World Reference Center for Emerging Viruses and Arboviruses (WRCEVA) in Galveston, TX, which came at a reported concentration of 6 ϫ 10 4 PFU/l or approximately 6 ϫ 10 7 genomic equivalents/l of extracted RNA. Using gBlock standards, we confirmed the viral copy number to be within 3-fold of the estimated value of 6 ϫ 10 7 PFU/l, which is consistent with expected variation from freeze-thaw and qRT-PCR. Quantitative standards were prepared from the WRCEVA sample by serially diluting SARS-CoV-2 RNA (3 ϫ 10 5 , 3 ϫ 10 4 , 3 ϫ 10 3 , 300, 30, and 3 genomic equivalent copies per RT-PCR) and analyzed by SARS-CoV-2 detection assays using the Corman E-gene and CDC N2 primer-probe sets. In parallel, 10-fold serial dilutions of RNA from the SC5688 clinical sample were prepared and tested using the same assays.

The standard curve generated from the quantitative standards was used to determine the number of viral genomic equivalents per microliter in the SC5688 sample.

The first two primer-probes sets that we evaluated were the UW RdRp and the Corman N-gene sets. Assays using these primer-probe sets were used to examine approximately 300 clinical samples of unknown SARS-CoV-2 status. RNA was extracted for these assays using two different systems. The first was the MagNA Pure LC (LC) system, which is able to process 32 samples at a time and elutes RNA into 200 l of buffer. The second was the MagNA Pure 96 (MP96) system, which is able to process 96 samples at a time and elutes RNA into 100 l of buffer. One sample (SC5688) out of the 300 was positive for SARS-CoV-2. SC5688 was positive for SARS-CoV-2 regardless of which of the two RNA extraction methods and which of the two primer-probe sets were used.

Sample SC5688 was determined to have a concentration of SARS-CoV-2 genomic equivalents of 3.16 ϫ 10 5 equivalents/l. We used sample SC5688 to determine the limit of detection (LOD) of assays using the UW RdRp and Corman N-gene primer-probe sets. For both of these primer-probe sets, 20 out of 20 replicate assays were still positive for SARS-CoV-2 when SC5688 was diluted to 1:2 ϫ 10 3 , the equivalent of about 790 genomic equivalents per reaction. Finally, we tested the specificity of assays using both primer-probe sets by running them on a collection of samples that are positive for respiratory viruses other than SARS-CoV-2. Assays using both sets were found to be 100% specific with no false positives noted in this analysis.

Assays using the Corman RdRp and E-gene sets were found to have LODs of about 316 viral genomic equivalents per reaction. We next tested assays using the Corman RdRp and Corman E-gene primer-probe sets. We again used two RNA extraction methods. The LC extraction method was performed using the same protocol as before. However, for the MP96 method, we eluted RNA into 50 l of buffer instead of 100 l of buffer to determine whether this would increase the sensitivity of SARS-CoV-2 detection assays. We ran assays using both the Corman RdRp and the Corman E-gene primer-probe sets coupled with both RNA extraction methods on 10 samples confirmed by the WSDOH Public Health Laboratories to be positive for SARS-CoV-2. The results of these tests are shown in Table 1 . There was one sample that was positive for assays using the E-gene set when the MP96 system was used but not when the LC system was used, and there were two samples that were positive for assays using the RdRp set when the MP96 system was used but not when the LC system was used. Based on these results, we subsequently used the MP96 RNA extraction system with RNA eluted into 50 l of buffer for all analyses. To assess the sensitivity of assays using the Corman RdRp and E-gene primer-probe sets, we ran them on dilutions of sample SC5688. For both of these primer-probe sets, 20 out of 20 replicate assays were still positive for SARS-CoV-2 when SC5688 was diluted to 1:2 ϫ 10 4 , the equivalent of about 316 genomic equivalents per reaction. We also tested the specificities of assays using these sets by running them on our collection of samples positive for various respiratory viruses. Like assays using the UW RdRp and the Corman N-gene primers and probes, those using the Corman RdRp and E-gene sets were 100% specific with no false positives noted.

Assays using the CDC N1 and N2 primer-probe sets performed better than those using the N3 set. Given that assays using the Corman RdRp and E-gene primer-probe sets were more sensitive than those using the UW RdRp and Corman N-gene sets, we wanted to compare the former to assays using the primer-probe sets published by the CDC: CDC N1, CDC N2, and CDC N3. We ran assays using these three sets on the 10 positive samples obtained from the WSDOH Public Health Laboratories. The results of these analyses are shown in Table 2 . All assays produced positive results for all samples, except for the assay using the CDC N3 primers-probe, which produced a negative result for SC5784. The assay using the Corman RdRp set also produced a negative result for this sample. For the other nine samples, assays using the Corman RdRp set consistently produced the highest cycle thresholds out of all the assays compared in Table 2 followed by assays using the Corman E-gene set.

Assays using the CDC N2 and Corman E-gene primer-probe sets were more sensitive than those using the CDC N1 and Corman RdRp sets and the BGI kit. Our final analysis was to test the sensitivities and specificities of assays using the CDC primer-probe sets and compare these to the sensitivities and specificities of assays using the Corman RdRp and E-gene sets. Because assays using the N3 set did not perform as well as those using the N1 and N2 sets, we did not include the former set in this analysis. We did, however, include in this analysis an evaluation of the sensitivity and specificity of assays performed using a SARS-CoV-2 test kit from BGI. To directly compare LODs of assays using the Corman RdRp and E-gene sets to those using the N1 and N2 sets and the BGI kit, we ran assays on dilutions ranging from 1:10 4 to 1:10 7 of sample SC5688. We again ran 20 duplicate assays with each primer-probe set and with the BGI kit on dilutions of SC5688 as listed in Table 3 . The least sensitive assays were the ones that used the Corman RdRp primer-probe set. At a dilution of 1:10 5 (the equivalent of 63 viral genomic equivalents per reaction), only 17 out of the 20 assays that used this set were positive for SARS-CoV-2. This set of outcomes is significantly different (Fisher's exact test, P value ϭ 7.1 ϫ 10 Ϫ3 ) from those observed for assays using the Corman E-gene, CDC N1, and CDC N2 primer-probe sets and the BGI kit, for which 20 out of 20 assays were positive. At a dilution of 1:10 6 (the equivalent of about 6.3 genomic equivalents per reaction), assays using the CDC N2 and the Corman E-gene sets performed equally well (Fisher's exact test, P value ϭ 0.3), with 18 and 17 replicate assays resulting as positive, respectively. The number of replicates that were positive was significantly higher (Fisher's exact test, P value ϭ 2.5 ϫ 10 Ϫ6 ) for assays using these two primer-probe sets relative to assays using the CDC N1 and Corman RdRP and the BGI kit when assays were run on a 1:10 6 dilution of sample SC5688. The specificities of assays using the CDC N1 and N2 primer-probe sets and the BGI kit were also tested using our panel of samples positive for various respiratory viruses. Assays using the N1 and N2 sets and the BGI kit were found to be 100% specific.

Known cases of COVID-19 have now exceeded 375,000 worldwide. While the number of new infections in China appears to be leveling off with fewer identified each day, the number of cases continues to rapidly increase in other nations across the world (3). In the coming days and weeks, many clinical laboratories will be developing and optimizing their own SARS-CoV-2 testing protocols. Maximizing the sensitivity and specificity of these tests is critical to efforts around the world to minimize the impact of this pandemic on global health.

A number of different primer-probe sets for use in SARS-CoV-2 detection assays and SARS-CoV-2 testing kits have been developed and are now available. As we have demonstrated here, the performance characteristics of assays using these primer-probe sets and testing kits are variable. Of the seven different primer-probe sets and one testing kit that we evaluated, all were found to be highly specific with no false-positive results observed when assays were run on samples positive for a number of other respiratory viruses. Variability was, however, observed in the sensitivities of these tests. We found assays using the CDC N2 and Corman E-gene primer-probe sets to be particularly sensitive. Assays using these sets were able to detect SARS-CoV-2 in 10 out of 10 known positive clinical samples. They were also able to reliably detect SARS-CoV-2 in a sample containing only about 6 genomic equivalents of viral RNA. In addition to our evaluation of different assays for SARS-CoV-2 detection, we also show that it is possible to significantly increase capacity for the RNA extraction step of SARS-CoV-2 testing without sacrificing sensitivity. In summary, we report variable performance characteristics of SARS-CoV-2 detection assays using seven different primer-probe sets and one complete testing kit when applied to clinical samples. While all assays evaluated were highly specific, some, such as those using the CDC N2 and the Corman E-gene sets, were found to be more sensitive than others. These findings will provide important insights on SARS-CoV-2 detection assay design to labs that are currently working to develop their own testing methods. Our results also emphasize the importance of ongoing optimization of viral detection assays following the emergence of novel viral pathogens.

Supplemental material is available online only. SUPPLEMENTAL FILE 1, PDF file, 0.9 MB.

Novel coronavirus (2019-nCoV) situation report-1. World Health Organization

The species Severe acute respiratory syndromerelated coronavirus: classifying 2019-nCoV and naming it SARS-CoV-2

An interactive web-based dashboard to track COVID-19 in real time

Early transmission patterns of coronavirus disease 2019 (COVID-19) in travellers from Wuhan to Thailand

First U.S. case of potentially deadly Chinese coronavirus confirmed in Washington State

Coronavirus reaches Federal Way as USPS employee tests positive

Coronavirus deaths in Washington now at 16, with 102 confirmed cases

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

-nCoV) real-time rRT-PCR panel primers and probes

Division of Viral Diseases. 2020. Real-time RT-PCR panel for detection of 2019-novel coronavirus: instructions for use

This work was support by the National Institute of Allergy and Infectious Diseases (NIAID) (grant number 5T32AI118690-04 T32 Training Grant to A.M.C. at the Fred Hutchinson Cancer Research Center).