key: cord-0989531-2bf3xzni authors: Zhen, Wei; Manji, Ryhana; Smith, Elizabeth; Berry, Gregory J title: Comparison of Four Molecular In Vitro Diagnostic Assays for the Detection of SARS-CoV-2 in Nasopharyngeal Specimens date: 2020-04-22 journal: nan DOI: 10.1101/2020.04.17.20069864 sha: 2c7654bf3edcb488a38b5d38989b109798e96031 doc_id: 989531 cord_uid: 2bf3xzni The novel human coronavirus SARS-CoV-2 was first discovered in the city of Wuhan, Hubei province, China, causing an outbreak of pneumonia in January 2020. As of April 10, 2020, the virus has rapidly disseminated to over 200 countries and territories, causing more than 1.6 million confirmed cases of COVID-19 and 97,000 deaths worldwide. The clinical presentation of COVID-19 is fairly non-specific, and symptoms overlap with other seasonal respiratory infections concurrently circulating in the population. Further, it is estimated that up to 80% of infected individuals experience mild symptoms or are asymptomatic, confounding efforts to reliably diagnose COVID-19 empirically. To support infection control measures, there is an urgent need for rapid and accurate molecular diagnostics to identify COVID-19 positive patients. In the present study, we have evaluated the analytical sensitivity and clinical performance of four SARS-CoV-2 molecular diagnostic assays granted Emergency Use Authorization by the FDA using nasopharyngeal swabs from symptomatic patients. This information is crucial for both laboratories and clinical teams, as decisions on which testing platform to implement are made. RNA internal control (Q670 probe) is used to detect RT-PCR failure and/or inhibition. The results 115 interpretation algorithm for reporting a positive specimen requires only one of the two targets to be 116 detected (S or ORF1ab gene). 117 118 GenMark ePlex SARS-CoV-2 EUA panel. Testing with the ePlex SARS-CoV-2 panel was performed 119 according to the manufacturer's instructions for use. Briefly, after vortexing for 3-5 seconds, 200 μl of the 120 primary NPS sample was aspirated into the sample delivery device (SDD) provided with the ePlex SARS-121 CoV-2 panel kit and vortexed once again for 10 seconds. The entire volume of the SDD was dispensed into 122 the sample loading port of the SARS-CoV-2 test cartridge, followed by firmly pushing down on the cap to 123 securely seal the sample delivery port. Each cartridge was bar-coded and scanned with the ePlex 124 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 22, 2020. Internal Control-S (IC-S) was added to each test specimen and controls via the working Panther Fusion 132 Capture Reagent-S (wFCR-S). Hybridized nucleic acid was then separated from the specimen in a magnetic 133 field. After wash steps, the elution step occurs and outputs 50µL of purified RNA. Then 5µL of eluted 134 nucleic acid is transferred to a Panther Fusion reaction tube already containing oil and reconstituted 135 mastermix. The Panther Fusion® SARS-CoV-2 assay amplifies and detects two conserved regions of the 136 ORF1ab gene in the same fluorescence channel. The two regions are not differentiated and amplification 137 of either or both regions leads to a fluorescent ROX signal. The results interpretation algorithm for 138 reporting a positive specimen requires only one of the two targets to be detected (ORF1a or ORF1b gene). 139 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Analytical sensitivity per claim 500 copies /mL 500 copies /mL 100,000 copies /mL 1x10 -2 TCID50/mL 141 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Cohen's kappa values (κ) were also calculated as a measure of overall agreement, with values categorized 164 as almost-perfect (>0.90), strong (0.80 to 0.90), moderate (0.60 to 0.79), weak (0.40 to 0.59), minimal 165 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 22, 2020. . https://doi.org/10.1101/2020.04.17.20069864 doi: medRxiv preprint (0.21 to 0.39), or none (0 to 0.20) (12-13). Probit analyses were used for the copies/mL determination of 166 the analytical sensitivity study. The dose-response 95th percentile (with 95% confidence interval [CI]) 167 model was assessed using the Finney and Stevens calculations (14) . 168 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Analytical Sensitivity. A serial dilution panel of SARS-CoV-2 control was tested to determine the LoD, 170 defined as the minimum concentration with detection of 100% by positive rate and 95% by Probit analysis. 171 The LoD established by percent positive rate ranged from 1,000 copies/mL by both the GenMark and the 172 modified CDC assays to 50 copies/mL by the DiaSorin Molecular assay ( Table 2) . The LoD results were 173 further subjected to Probit analysis. The 95% detection limit of the CDC assay was 779 ± 27 copies/mL for 174 the N1 gene and 356 ± 20 copies/mL for the N2 gene. For the DiaSorin Molecular assay, the 95% detection 175 limit was 39 ± 23 copies/mL for the S gene and 602 ± 28 copies/mL for ORF1ab. For the Hologic assay, the 176 95% detection limit was 83 ± 36 copies/mL for ORF1ab. Probit analysis could not be performed for the 177 GenMark assay ( Table 2) . The final LoD, according to the assay results interpretation algorithm from each 178 manufacturer, ranged from 1,000 copies/mL by the GenMark assay to 39 ± 23 copies/mL by the DiaSorin 179 Molecular assay ( Table 2) . 180 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Table 3) . 192 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Details for discordant sample analysis are shown in Table 4 . A total of five discordant samples were found 203 among three out of the four platforms. One false positive sample (NW-104) had Ct values of 38.9 and 39.6 204 for N1 and N2 genes, respectively, on initial testing by the modified CDC assay. After repeating extraction 205 and retesting, the sample was determined to be negative. Two samples (NW-97 & NW-99) were 206 considered false negative by GenMark but positive by the other three methods. After reprocessing and 207 retesting, the GenMark assay was able to detect both samples as positive. Two additional false positive 208 samples (NW-83 & NW-85) were found by the Hologic assay; original samples were retested and were 209 found to be positive and negative, respectively. Following retesting of the five discordant samples, the 210 GenMark ePlex SARS-CoV-2 EUA panel showed an improvement of sensitivity to 100% (51/51). 211 Additionally, a 100% specificity (53/53) was obtained for the CDC assay, while Hologic improved to 98% 212 (52/53) ( Table 4) . 213 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 22, 2020. Table 5 . The HOT ranged between all platforms. The longest hands-219 on time was the Hologic assay at ~2 hours followed by the modified CDC assay with ~1 hour and 30 220 minutes. Very comparable HOT was found for DiaSorin Molecular and GenMark with a range of 16 221 minutes and 12 minutes, respectively. The run time averaged 90 minutes for modified CDC, DiaSorin 222 Molecular, and GenMark. Hologic was the exception with 4 hours with 35 mins of run time ( Table 5) . 223 Overall turn-around time assessment, from sample to results, showed DiaSorin Molecular with the least 224 overall turn-around time to results, followed by GenMark, modified CDC assay and Hologic with the 225 greatest overall time ( Table 5) . 226 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 22, 2020. Our data suggest that all four PCR methods yielded comparable results (κ ≥ 0.96); however, we 250 did observe a notable difference in the sensitivity of the methods during this large-scale evaluation of EUA 251 in vitro diagnostic assays. Our study showed that the DiaSorin Molecular and Hologic Fusion assays out-252 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted April 22, 2020. . https://doi.org/10.1101/2020.04.17.20069864 doi: medRxiv preprint performed both the modified CDC and GenMark assays when it came to overall LoD, with GenMark having 253 the overall highest LoD of all four platforms evaluated. DiaSorin Molecular had the lowest LoD (39 ± 23 254 copies/mL), closely followed by Hologic (83 ± 36 copies/mL). The modified CDC assay showed a final LoD, 255 of 779 ± 27 copies/mL based on the results interpretation algorithm. It is worth mentioning that this assay 256 requires both targets to be fully detected, thus clinical samples falling in this concentration range would 257 be identified and repeated, potentially generating additional turnaround time and laboratory labor. In 258 contrast, GenMark could only detect 100% of replicates at 1,000 copies/mL, and was not able to reliably 259 detect replicates below 1,000 copies/mL, thus patient specimens below this concentration range could 260 potentially be missed. One important limitation to mention is that sensitivity using Probit analysis could 261 not be calculated for GenMark since Ct values are not available as part of the ePlex system result 262 interpretation. 263 The clinical correlation was also consistent with LoD findings, where both the DiaSorin Molecular 265 and Hologic assays had 100% sensitivity and detected all specimens deemed positive by the consensus 266 standard (interpretation of three of four evaluated assays as "gold standard"), whereas GenMark missed 267 two positive specimens (which were subsequently detected by GenMark upon repeat). DiaSorin 268 Molecular and GenMark showed 100% specificity, while Hologic and the CDC assay initially had two and 269 one discordant results, respectively. Repeat testing of these three specimens showed that for Hologic, 270 NW-83 repeated as positive a second time and was therefore potentially a false positive and NW-85 was 271 negative upon repeat, meaning this result could have previously been a false positive as well. The 272 continued discordant result from NW-83 could potentially be attributed to specificity issues, since the 273 assay did not prove to be the most sensitive assay among those used in the study, with DiaSorin Molecular 274 showing a slightly lower LoD. Repeat testing of NW-104 on the modified CDC assay was negative. 275 Considering the LoD of the modified CDC assay, coupled with the fact that both DiaSorin Molecular and 276 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. When it comes to the hands-on and turnaround time of the four assays in this study, the 281 throughput and workflow evaluation are clearly shown in Table 5 and are based on lab technologist 282 experience in our laboratory. As a routine real-time RT-PCR assay, the modified CDC requires nucleic acid 283 extraction, master mix preparation and PCR setup, standard PCR amplification, as well as interpretation to purchase additional instruments to allow for testing of more samples at a time in order to satisfy patient 289 testing volume requirements. The Hologic Panther Fusion platform has more of an automated workflow, 290 with five samples processed at a time after loading. The sample to answer time for the first five samples 291 is 2 hours and 40 min, followed by 5 results every 5 minutes after loading 120 samples; the total assay run 292 time for 120 specimens is approximately 4 hours and 35 min. It is also important to note that the Hologic 293 platform has longer hands-on time, since the technologist has to load the primers, probes, and other 294 consumables and the fact that 120 clinical samples have to be manually transferred to Sample Lysis Buffer 295 tubes. These steps, especially the pipetting of the specimen into the lysis tube, can be somewhat labor 296 intensive and time consuming, bumping the overall turnaround time for 120 specimens closer to the 7 297 hour mark. It is important to emphasize that each platform has their advantages. For workflow, TAT, and 298 ease of use, the three sample-to-answer platforms (DiaSorin Molecular, Hologic, GenMark) out-299 performed the modified CDC assay, which is a manual assay requiring many steps, specialized personnel, 300 Coronavirus disease (COVID-19) outbreak. World Health Organization Locations with confirmed COVID-19 cases, global map Coronavirus COVID-19 Global Cases by the Center for Systems Science and Engineering COVID-19: Zoonotic aspects Receptor Recognition by the Novel Coronavirus from 343 Wuhan: an Analysis Based on Decade-Long Structural Studies of SARS Coronavirus SARS-CoV-2 Cell Entry Depends on ACE2 and 346 TMPRSS2 and Is Blocked by a Clinically Proven Protease Inhibitor Tissue distribution of ACE2 348 protein, the functional receptor for SARS coronavirus. A first step in understanding SARS 349 pathogenesis Report of the WHO-China Joint Mission on Coronavirus Disease Aerosol and surface stability of HCoV-19 (SARS-CoV-2) compared to 354 SARS-CoV-1 The measurement of observer agreement for categorical data Interrater reliability: the kappa statistic A table for the calculation of working probits and weights in 362 probit analysis Covid-19 -Navigating the Uncharted SARS-CoV-2 Viral Load in Upper Respiratory Specimens of Infected 366 Groups at Higher Risk for Severe Illness We would also like to thank Diasorin Molecular LLC. for providing the reagents used in this study. 328Gregory Berry has previously given education seminars for Hologic, Inc., and received an Honorarium. 329 All rights reserved. No reuse allowed without permission.(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. This study has several limitations that should be mentioned. First, this was a single center study 305 and the majority of the specimens were frozen after initial testing on the GenMark assay. While these 306 limitations are present, they have been minimized by the fact that GenMark assay (which was the least 307 sensitive platform in the analysis) actually had a potential competitive advantage, since it was the assay 308 initially performed on fresh specimens. Second, while the number of specimens included in the clinical 309 correlation was only 104, the patient samples spanned the entire range of clinical positives and reflected 310 our overall true positivity rate, which was between 50-60% during this time period of the COVID-19 311 outbreak. 312 In summary, we have evaluated four molecular in vitro diagnostic assays for the qualitative 314 detection of SARS-CoV-2 in nasopharyngeal specimens. The data from our evaluation suggest that the 315 modified CDC, DiaSorin Molecular, Hologic and GenMark assays performed similarly (κ ≥ 0.96) and that all 316 but the CDC assay can function in a sample-to-answer capacity. The GenMark assay, however, was less 317 sensitive and had a higher LoD than both the DiaSorin Molecular and Hologic assays. When considering 318 the design of all four assays, differences that could affect assay performance could include characteristics 319 such as input volume of initial specimen, RNA purification and elution volume differences, and overall 320 differences in gene targets. The DiaSorin Molecular platform has lower testing volume capability 321 compared to the Hologic assay (8 specimens/disc run vs. 120 specimens loaded at once), but has a faster 322 TAT and less reagent/sample preparation. All of these parameters, along with patient care needs, may 323 All rights reserved. No reuse allowed without permission.(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint this version posted April 22, 2020. . https://doi.org/10.1101/2020.04.17.20069864 doi: medRxiv preprint assist clinical laboratories to identify and choose the correct testing platform that best fits their needs for 324 the diagnosis of patients infected with this novel human coronavirus. 325 326 All rights reserved. No reuse allowed without permission.(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint this version posted April 22, 2020. . https://doi.org/10.1101/2020.04.17.20069864 doi: medRxiv preprint All rights reserved. No reuse allowed without permission.(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. All rights reserved. No reuse allowed without permission.(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint this version posted April 22, 2020. . https://doi.org/10.1101/2020.04.17.20069864 doi: medRxiv preprint