key: cord-0963321-dpk3fqqj
authors: van Kasteren, Puck B.; van der Veer, Bas; van den Brink, Sharon; Wijsman, Lisa; de Jonge, Jørgen; van den Brandt, Annemarie; Molenkamp, Richard; Reusken, Chantal B.E.M.; Meijer, Adam
title: Comparison of commercial RT-PCR diagnostic kits for COVID-19
date: 2020-04-24
journal: bioRxiv
DOI: 10.1101/2020.04.22.056747
sha: ec17e5f3c5a412d7716b0ae356de61e4bafb0cf9
doc_id: 963321
cord_uid: dpk3fqqj

The final months of 2019 witnessed the emergence of a novel coronavirus in the human population. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has since spread across the globe and is posing a major burden on society. Measures taken to reduce its spread critically depend on timely and accurate identification of virus-infected individuals by the most sensitive and specific method available, i.e. real-time reverse transcriptase PCR (RT-PCR). Many commercial kits have recently become available, but their performance has not yet been independently assessed. The aim of this study was to compare basic analytical and clinical performance of selected RT-PCR kits from seven different manufacturers (Altona Diagnostics, BGI, CerTest Biotec, KH Medical, PrimerDesign, R-Biopharm AG, and Seegene). We used serial dilutions of viral RNA to establish PCR efficiency and estimate the 95% limit of detection (LOD95%). Furthermore, we ran a panel of SARS-CoV-2-positive clinical samples (n=16) for a preliminary evaluation of clinical sensitivity. Finally, we used clinical samples positive for non-coronavirus respiratory viral infections (n=6) and a panel of RNA from related human coronaviruses to evaluate assay specificity. PCR efficiency was ≥96% for all assays and the estimated LOD95% varied within a 6-fold range. Using clinical samples, we observed some variations in detection rate between kits. Importantly, none of the assays showed cross-reactivity with other respiratory (corona)viruses, except as expected for the SARS-CoV-1 E-gene. We conclude that all RT-PCR kits assessed in this study may be used for routine diagnostics of COVID-19 in patients by experienced molecular diagnostic laboratories.

Coronavirus disease 2019 is caused by the severe acute respiratory syndrome coronavirus 25 2 (SARS-CoV-2). This virus emerged in the human population in the final months of 2019 from a, so far 26 unidentified, animal reservoir and has since spread across the globe (1). The SARS-CoV-2 pandemic 27 poses an enormous burden on society, economic and healthcare systems worldwide, and various 28 measures are being taken to control its spread. Many of these measures critically depend on the timely 29 and accurate diagnosis of virus-infected individuals. Real-time reverse transcription polymerase chain 30 reaction (RT-PCR) is the most sensitive and specific assay and therefore preferred (2, 3). Whereas 31 many COVID-19 RT-PCR kits are currently commercially available, an independent assessment of these 32 products is not yet publicly available and direly needed to guide implementation of accurate tests in 33 a diagnostic market that is flooded with new tests. As of 11 April 2020, the FIND organization listed 34 201 molecular assays on their website as being on the market (www.finddx.org/covid-19/pipeline). 35

Coronaviruses are positive-stranded RNA viruses that express their replication and 36 transcription complex, including their RNA-dependent RNA polymerase (RdRp), from a single, large 37 open reading frame referred to as ORF1ab (4). The coronavirus structural proteins, including the 38 envelope (E), nucleocapsid (N), and spike (S) proteins, are expressed via the production of subgenomic 39 messenger RNAs, which during certain stages of the replication cycle far outnumber (anti)genomic 40

RNAs. The ORF1ab/RdRp, E, N, and S genes are the targets most frequently used for SARS-CoV-2 41 detection by RT-PCR. For example, the "Corman" PCR, which was co-developed in our lab and is now 42 routinely used for our in-house diagnostic work, targets a combination of the E-gene and the RdRp-43 gene (2). In this set-up, the E-gene primer/probe set is specific for bat(-related) betacoronaviruses, 44 and therefore detects both SARS-CoV-1 and -2. In addition, whereas the RdRp-gene primers are also 45 specific for bat(-related) betacoronaviruses, two probes are used: one specific for bat (-related) 46 betacoronaviruses and another specific for SARS-CoV-2. In this study we only used the RdRp probe that is specific for SARS-CoV-2.

Here, we provide a comparison of a selection of seven readily available COVID-19 RT-PCR kits from 49 different manufacturers (Table 1) . One of these kits (BGI) was recently also included in a comparative 50 study of various SARS-CoV-2 primer/probe sets (5). Most of the selected kits are CE-IVD certified and 51 can be produced in large quantities. Using a dilution series of SARS-CoV-2 RNA we determine the 95% 52 limit of detection (LOD95%) for each of these assays. In addition, a concise panel of clinical samples 53 (n=22) was run to provide a first indication of clinical sensitivity and specificity. Although some kits 54 appeared to perform better than others at identifying clinical samples at very low concentrations of 55 SARS-CoV-2 RNA, all tests were able to identify positive samples with Ct≤34.5 in our in-house E-gene 56 PCR. Therefore, we conclude that all of the RT-PCR kits assessed in this study may be used for routine 57 diagnostics of COVID-19 by experienced molecular diagnostic laboratories. 58

Commercially available COVID-19 RT-PCR kits were identified via the FindDx website 61 (www.finddx.org/covid-19/pipeline, March 2020) and requests for information and sample kits were 62 sent via e-mail to approximately 20 manufacturers and/or distributors, focusing on those kits that had 63 already obtained CE-IVD certification. Promising commercial kits were selected based on: 1) listing on 64 the FindDx website; 2) responsiveness to requests; 3) accessible information (in English); 4) 65 compatibility with different PCR platforms; 5) considerable production capacity. Notably, all of the 66 PCR kits that we had selected for our analysis have in the meantime also been selected for the first 67 round of independent evaluation by FIND (www.finddx.org/covid-19/sarscov2-eval-molecular/, April 68 2020). All of the kits included in our analysis were provided free of charge and none of the 69 manufacturers were involved in the assessment and interpretation of the results. The selection 70 encompasses both kits that require transport and storage at 71 -20°C and kits that can be transported and stored at room temperature. Target genes for each RT-PCR 72 kit were available in the assay documentation or upon request (for an overview, see Table 1 ). All PCRs were run on a LightCycler 480 II (LC480II, Roche) and performed according to the manufacturer's 74 instructions for use. Of note however, for some kits (BGI, KH Medical, and Seegene) settings for the 75 LC480II were not provided and were therefore adapted from those provided for another machine. 76

To establish PCR efficiency we first ran a duplicate 10-fold dilution series of viral RNA for each assay. 78

RNA was isolated from SARS-CoV-2 viral particles (hCoV-79 19/Netherlands/Diemen_1363454/2020, GISAID: EPI_ISL_413570) obtained from cell culture using 80 the MagNA Pure LC Total Nucleic Acid Isolation Kit (Roche). We determined the slope by linear 81 regression in GraphPad Prism and defined the required levels for PCR efficiency (E) and R 2 as >95% 82 and >0.95, respectively. Next, we ran four replicates of a 2-fold dilution series (diluted in yeast carrier 83 RNA in water) to determine the LOD95% by Probit analysis using SPSS Statistics (IBM, version 24). The 84 limited range of the dilution series did not allow for determination of a confidence interval for the 85 LOD95% for all assays, which should therefore be regarded as an approximation and not considered 86 definitive. The starting concentration of the viral RNA (copies/ml) was determined by digital PCR 87 targeting the SARS-CoV-2 RdRp-gene and was specific for the positive sense genomic RNA (2). 88

Finally, a panel of clinical samples with in-house confirmed SARS-CoV-2 (17.25≤Ct≤39.6 for the E-gene 90 during initial diagnostics; n=16) or other respiratory viruses (influenza virus type A (n=2), rhinovirus 91 (n=2), RSV-A and -B) was prepared (for Ct values obtained in initial diagnostics, see supplementary 92 Table S1 ). RNA was isolated anew from stored clinical samples (naso-and/or oropharyngeal swabs in 93 GLY-medium) using the MagNA Pure 96 DNA and Viral NA Small Volume Kit (Roche) and was assessed 94 with a single replicate to obtain a first indication of clinical specificity and sensitivity. No re-test was 95 performed when the result was inconclusive according to the manufacturer's instructions for 96 interpretation of the result (n=2). In addition to clinical samples, a panel of viral RNA from related cell 97 cultured human coronaviruses (including SARS1, MERS, NL63, OC43, and 229E) was used to assess 98 cross-reactivity within the coronavirus family (for Ct values of these samples see supplementary Table  99 S1). 100

PCR efficiency was above the required level for all kits included in the study. We first assessed PCR 102 efficiency for each target gene assay by running a duplicate 10-fold dilution series of SARS-CoV-2 viral 103 RNA ( Figure 1 ). All assays showed an efficiency ≥96% and R squares were ˃0.97, which are both well 104 above the pre-defined required level. Since the applied filter settings were not correct for reading the 105 Seegene N-gene assay, we excluded these data from all of our analyses. 106

The LOD95% varied within a 6-fold range between the kits included in the study. The 10-fold dilution 107 series provided a first indication of the LOD95% for each assay and were used to determine the starting 108 point of a 2-fold dilution series performed with four replicates to come to a more precise estimate (for 109

Ct values, see supplementary Table S2 ). Probit analysis was performed to estimate the LOD95%, which 110 is shown in Table 2 . Notably, due to the limited extent of the dilution series, this analysis did not always 111 provide upper and lower bounds of the estimate and should not be considered definitive. We found 112 that the estimated LOD95% for the various targets of the RT-PCR kits varied within a 6-fold range, with 113 the RT-PCR kit from Altona Diagnostics having the lowest LOD95% at 3.8 copies/ml for both the E-and 114 S-gene assays and the PrimerDesign kit having the highest LOD95% at 23 copies/ml (Table 2) . Overall, 115 our in-house "Corman" RT-PCR had the lowest estimated LOD95% at 0.91 copies/ml for the E-gene 116 assay (2). 117

The clinical sensitivity appears to vary between the kits included in the study. Next, we analyzed a 118 panel of clinical samples previously submitted for routine SARS-CoV-2 diagnostics (n=16) for which the 119 presence of various amounts of SARS-CoV-2 RNA had been confirmed using our in-house PCR. In compared to the initial diagnostic results for our in-house E-gene PCR. For this reason, even using our 124 in-house PCR we could not confirm the presence of SARS-CoV-2 RNA in 3 out of 16 samples (see Figure  125 2A and supplementary Table S1 ). The positive identification rate for the various RT-PCR kits varied 126 from 10 to 13 out of 16 samples (Figure 2A) , with R-Biopharm AG performing best (13/16), followed 127 by BGI, KH Medical, and Seegene (12/16), CerTest BioTec (11/16), and Altona Diagnostics and 128

PrimerDesign (10/16). Of note, Seegene had one "inconclusive" sample according to the 129 manufacturer's instructions for interpretation, which might have tested positive upon re-testing but 130 has now been counted as "negative". All target gene assays were able to positively identify the 10 131 clinical samples with the highest concentrations of SARS-CoV-2 (Ct≤34.50 in our in-house E-gene PCR). Table S1 ). We also ran a panel consisting of 138 cell culture-derived viral RNA for related human coronaviruses (SARS1, MERS, NL63, OC43, and 229E) 139 to check for cross-reactivity within the coronavirus family. Of these, only the SARS-CoV-1 E-gene was 140 identified, as per design, by assays from Altona Diagnostics, Seegene, and our in-house PCR 141 (Supplementary Table S1 ). 142

Here we provide a comparison of seven commercially available RT-PCR kits for the detection of SARS-144

CoV-2 in clinical samples. All RT-PCR kits performed satisfactorily regarding PCR efficiency (≥96%) and 145 the estimated LOD95% varied within a 6-fold range between kits (3.8-23 copies/ml). Notably, the copy 146 number concentration of the standard was determined by digital PCR on the positive sense RdRp gene 147 and therefore provides an indication of the number of viral particles per ml. The actual copy number 148 for each RT-PCR target and accompanying limit of detection may vary depending on, for example, the 149 amount of subgenomic messenger RNA-containing cells that are present in the (clinical) Tables 218 Table 1 . Overview of kits for RT-PCR-based detection of SARS-COV-2 included in the study. 

A pneumonia outbreak associated 178 with a new coronavirus of probable bat origin

Detection of 2019 180 novel coronavirus (2019-nCoV) by real-time RT-PCR

Laboratory 182 readiness and response for novel coronavirus (2019-nCoV) in expert laboratories in 30

A contemporary view of coronavirus transcription

Performance of SARS-CoV-2 Detection Assays using Seven Different Primer/Probe Sets and 188

One Assay Kit

SARS-CoV-2 Viral Load in Upper 190

Respiratory Specimens of Infected Patients

We kindly thank Barbara Favié (RIVM, Bilthoven) for performing the digital PCR. In addition, we would 172 like to thank all manufacturers who kindly donated their RT-PCR kits for our evaluation. 173

This work was funded by the Dutch ministry of health, welfare, and sports (VWS). The RT-PCR kits 175 included in this study were provided free of charge. 176