key: cord-0774202-7d7i3cc8
authors: van Kasteren, Puck B.; van der Veer, Bas; van den Brink, Sharon; Wijsman, Lisa; de Jonge, Jørgen; van den Brandt, Annemarie; Molenkamp, Richard; Reusken, Chantal B.E.M.; Meijer, Adam
title: Comparison of commercial RT-PCR diagnostic kits for COVID-19
date: 2020-05-08
journal: J Clin Virol
DOI: 10.1016/j.jcv.2020.104412
sha: e8ad5180fc8bf15ea168c8d7d03b5707afe5efb3
doc_id: 774202
cord_uid: 7d7i3cc8

Abstract The final months of 2019 witnessed the emergence of a novel coronavirus in the human population. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has since spread across the globe and is posing a major burden on society. Measures taken to reduce its spread critically depend on timely and accurate identification of virus-infected individuals by the most sensitive and specific method available, i.e. real-time reverse transcriptase PCR (RT-PCR). Many commercial kits have recently become available, but their performance has not yet been independently assessed. The aim of this study was to compare basic analytical and clinical performance of selected RT-PCR kits from seven different manufacturers (Altona Diagnostics, BGI, CerTest Biotec, KH Medical, PrimerDesign, R-Biopharm AG, and Seegene). We used serial dilutions of viral RNA to establish PCR efficiency and estimate the 95% limit of detection (LOD95%). Furthermore, we ran a panel of SARS-CoV-2-positive clinical samples (n = 13) for a preliminary evaluation of clinical sensitivity. Finally, we used clinical samples positive for non-coronavirus respiratory viral infections (n = 6) and a panel of RNA from related human coronaviruses to evaluate assay specificity. PCR efficiency was ≥96% for all assays and the estimated LOD95% varied within a 6-fold range. Using clinical samples, we observed some variations in detection rate between kits. Importantly, none of the assays showed cross-reactivity with other respiratory (corona)viruses, except as expected for the SARS-CoV-1 E-gene. We conclude that all RT-PCR kits assessed in this study may be used for routine diagnostics of COVID-19 in patients by experienced molecular diagnostic laboratories.

2 available, i.e. real-time reverse transcriptase PCR (RT-PCR). Many commercial kits have recently become available, but their performance has not yet been independently assessed.

The aim of this study was to compare basic analytical and clinical performance of selected RT-PCR kits from seven different manufacturers (Altona Diagnostics, BGI, CerTest Biotec, KH Medical, PrimerDesign, R-Biopharm AG, and Seegene).

We used serial dilutions of viral RNA to establish PCR efficiency and estimate the 95% limit of detection (LOD95%). Furthermore, we ran a panel of SARS-CoV-2-positive clinical samples (n=13) for a preliminary evaluation of clinical sensitivity. Finally, we used clinical samples positive for noncoronavirus respiratory viral infections (n=6) and a panel of RNA from related human coronaviruses to evaluate assay specificity.

PCR efficiency was ≥96% for all assays and the estimated LOD95% varied within a 6-fold range. Using clinical samples, we observed some variations in detection rate between kits. Importantly, none of the assays showed cross-reactivity with other respiratory (corona)viruses, except as expected for the SARS-CoV-1 E-gene.

We conclude that all RT-PCR kits assessed in this study may be used for routine diagnostics of COVID-19 in patients by experienced molecular diagnostic laboratories. poses an enormous burden on society, economic and healthcare systems worldwide, and various measures are being taken to control its spread. Many of these measures critically depend on the timely and accurate diagnosis of virus-infected individuals. Real-time reverse transcription polymerase chain reaction (RT-PCR) is the most sensitive and specific assay and therefore preferred (2, 3) . Whereas many COVID-19 RT-PCR kits are currently commercially available, an independent assessment of these products is not yet publicly available and direly needed to guide implementation of accurate tests in a diagnostic market that is flooded with new tests. As of 11 April 2020, the FIND organization listed 201 molecular assays on their website as being on the market (www.finddx.org/covid-19/pipeline).

Coronaviruses are positive-stranded RNA viruses that express their replication and transcription complex, including their RNA-dependent RNA polymerase (RdRp), from a single, large open reading frame referred to as ORF1ab (4). The coronavirus structural proteins, including the envelope (E), nucleocapsid (N), and spike (S) proteins, are expressed via the production of subgenomic messenger RNAs, which during certain stages of the replication cycle far outnumber (anti)genomic RNAs. The ORF1ab/RdRp, E, N, and S genes are the targets most frequently used for SARS-CoV-2 detection by RT-PCR. For example, the "Corman" PCR, which was co-developed in our lab and is now routinely used for our in-house diagnostic work, targets a combination of the E-gene and the RdRpgene (2) . In this set-up, the E-gene primer/probe set is specific for bat(-related) betacoronaviruses, and therefore detects both SARS-CoV-1 and -2. In addition, whereas the RdRp-gene primers are also specific for bat(-related) betacoronaviruses, two probes are used: one specific for bat (-related) betacoronaviruses and another specific for SARS-CoV-2. In this study we only used the RdRp probe that is specific for SARS-CoV-2.

Here, we provide a comparison of a selection of seven readily available COVID-19 RT-PCR kits from different manufacturers ( Table 1) . One of these kits (BGI) was recently also included in a comparative study of various SARS-CoV-2 primer/probe sets (5) . Most of the selected kits are CE-IVD certified and can be produced in large quantities. Using a dilution series of SARS-CoV-2 RNA we determine the 95% limit of detection (LOD95%) for each of these assays. In addition, a concise panel of clinical samples (n=22) was run to provide a first indication of clinical sensitivity and specificity. Although some kits appeared to perform better than others at identifying clinical samples at very low concentrations of SARS-CoV-2 RNA, all tests were able to identify positive samples with Ct≤34.5 in our in-house E-gene PCR. Therefore, we conclude that all of the RT-PCR kits assessed in this study may be used for routine diagnostics of COVID-19 by experienced molecular diagnostic laboratories.

Commercially available COVID-19 RT-PCR kits were identified via the FindDx website (www.finddx.org/covid-19/pipeline, March 2020) and requests for information and sample kits were sent via e-mail to approximately 20 manufacturers and/or distributors, focusing on those kits that had already obtained CE-IVD certification. Promising commercial kits were selected based on: 1) listing on the FindDx website; 2) responsiveness to requests; 3) accessible information (in English); 4) compatibility with different PCR platforms; 5) considerable production capacity. Notably, all of the PCR kits that we had selected for our analysis have in the meantime also been selected for the first round of independent evaluation by FIND (www.finddx.org/covid-19/sarscov2-eval-molecular/, April 2020). All of the kits included in our analysis were provided free of charge and none of the manufacturers were involved in the assessment and interpretation of the results. The selection encompasses both kits that require transport and storage at -20°C and kits that can be transported and stored at room temperature. Target genes for each RT-PCR kit were available in the assay documentation or upon request (for an overview, see Table 1 ). All PCRs J o u r n a l P r e -p r o o f were run on a LightCycler 480 II (LC480II, Roche) and performed according to the manufacturer's instructions for use. Of note however, for some kits (BGI, KH Medical, and Seegene) settings for the LC480II were not provided and were therefore adapted from those provided for another machine.

To establish PCR efficiency we first ran a duplicate 10-fold dilution series of viral RNA for each assay. (for Ct values of these samples see supplementary Table   S1 ).

PCR efficiency was above the required level for all kits included in the study. We first assessed PCR efficiency for each target gene assay by running a duplicate 10-fold dilution series of SARS-CoV-2 viral RNA ( Figure 1 ). All assays showed an efficiency ≥96% and R squares were ˃0.97, which are both well above the pre-defined required level. Since the applied filter settings were not correct for reading the Seegene N-gene assay, we excluded these data from all of our analyses.

The LOD95% varied within a 6-fold range between the kits included in the study. The 10-fold dilution series provided a first indication of the LOD95% for each assay and were used to determine the starting point of a 2-fold dilution series performed with four replicates to come to a more precise estimate (for Ct values, see supplementary Table S2 ). Probit analysis was performed to estimate the LOD95%, which is shown in Table 2 . Notably, due to the limited extent of the dilution series, this analysis did not always provide upper and lower bounds of the estimate and should not be considered definitive. We found that the estimated LOD95% for the various targets of the RT-PCR kits varied within a 6-fold range, with the RT-PCR kit from Altona Diagnostics having the lowest LOD95% at 3.8 copies/ml for both the E-and S-gene assays and the PrimerDesign kit having the highest LOD95% at 23 copies/ml (Table 2) . Overall, our in-house "Corman" RT-PCR had the lowest estimated LOD95% at 0.91 copies/ml for the E-gene assay (2) .

The clinical sensitivity appears to vary between the kits included in the study. Next, we analyzed a Table S1 ).

CoV-2 in clinical samples. All RT-PCR kits performed satisfactorily regarding PCR efficiency (≥96%) and the estimated LOD95% varied within a 6-fold range between kits (3.8-23 copies/ml). Notably, the copy As does the in-house "Corman" E-gene PCR, these E-gene assays are specific for bat(-related) betacoronaviruses, i.e. they detect both SARS-CoV-1 and -2. 2 According to manufacturer's website the kit is RUO, the FindDx website states CE-IVD certification for this kit. 3 Shipment is performed at RT 4 According to the manufacturer, CE-IVD certification will be applied for in the near future. The copy number was determined by digital PCR for the positive sense RdRp gene. Due to the limited range of the 2-fold dilution series, a confidence interval could not be determined for all assays. 2 The filter settings for the Seegene N-gene PCR were not correct and these results are therefore excluded. Abbreviations: E, envelope protein of SARS-CoV-2; LOD95%, 95% limit of detection; N, nucleocapsid protein of SARS-CoV-2; NA, not available; ORF, open reading frame; RdRp, RNA-dependent RNA polymerase of SARS-CoV-2; RT-PCR, reversetranscription polymerase chain reaction; S, spike protein of SARS-CoV-2; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

A pneumonia outbreak associated with a new coronavirus of probable bat origin

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

Laboratory readiness and response for novel coronavirus (2019-nCoV) in expert laboratories in 30 EU/EEA countries

A contemporary view of coronavirus transcription

Comparative Performance of SARS-CoV-2 Detection Assays using Seven Different Primer/Probe Sets and One Assay Kit

SARS-CoV-2 Viral Load in Upper Respiratory Specimens of Infected Patients

We kindly thank Barbara Favié (RIVM, Bilthoven) for performing the digital PCR. In addition, we would like to thank all manufacturers who kindly donated their RT-PCR kits for our evaluation.Declarations of interest: none.

This work was funded by the Dutch ministry of health, welfare, and sports (VWS). The RT-PCR kits included in this study were provided free of charge.