key: cord-0802148-tanp55ae
authors: Voelker, Chet R.; Ochoa, Anna R.; Armstrong-Spenrath, La'Quita; Lott, Lisa; McDaniel, Jennifer S.; Blackburn, August N.; Cornell, Lauren E.; Mahoney, Rachel; Asin, Susana N.
title: Evaluating Sensitivity and Specificity of the Biomeme Franklin(TM) three9 Real-Time PCR Device and SARS-CoV-2 Go-Strips Assay Using Clinical Samples
date: 2021-11-25
journal: J Clin Virol
DOI: 10.1016/j.jcv.2021.105046
sha: f695e6f3011b72a64a9af8f51987515d1a872d67
doc_id: 802148
cord_uid: tanp55ae

We evaluated the sensitivity and specificity of the Biomeme Franklin(TM) three9 Real-Time PCR Thermocycler and Biomeme SARS-CoV-2 Go-Strips in the detection of SARS-CoV-2. The Biomeme Franklin(TM) three9 platform is a portable, battery-operated system that could be used in remote settings. We assessed performance of the Biomeme SARS-CoV-2 detection system at a wide range of viral concentrations, examined cross-reactivity of the SARS-CoV-2 Go-Strips against several near-neighbor respiratory pathogens, and evaluated agreement against the BioFire® Respiratory Panel 2.1 in four clinical sample types. Our data indicate the Biomeme Go-Strips can reliably detect SARS-CoV-2 at a concentration of 4.2 × 10(3) copies/mL. No cross reactivity of the Go-Strips targets was detected against any of the tested near-neighbor respiratory pathogens. Cohen's kappa statistics ranged from 0.68 to 0.92 between results from the Biomeme SARS-CoV-2 Go-Strips and the BioFire® Respiratory Panel 2.1 in all the different sample types. Compared to the BioFire® Respiratory Panel 2.1, the Biomeme SARS-CoV-2 Go-Strips demonstrated statistically significantly lower sensitivity in 3 out of 5 sample types. Overall, our study demonstrates the Biomeme Franklin(TM) three9 used with the SARS-CoV-2 Go-Strips is an effective system for the detection of SARS-CoV-2 that could potentially be used in a remote or austere environment.

The World Health Organization declared Coronavirus Disease (COVID-19) a global pandemic on March 11 th , 2020 [1] . Since then, the pandemic has disrupted the lives of people worldwide and placed a significant burden on health care systems. As of August 4, 2021, more than 200 million COVID-19 cases have been confirmed worldwide, including approximately 4.7 million deaths [2] . The unprecedented global impact of the COVID-19 pandemic highlights the need for rapid, reliable diagnostic tests for the detection of SARS-CoV-2 that can lead to early intervention, improved health outcomes, and reduced spread of the disease.

Reverse Transcription-Polymerase Chain Reaction (RT-PCR) assays for the detection of viral RNA targets are the gold standard for SARS-CoV-2 diagnosis. Large healthcare facilities and laboratories in suburban and metropolitan areas have access to high-throughput, sample-to-answer RT-PCR platforms; however, more remote settings typically cannot access these technologies [3] . Rural areas, which on average, consist of older populations with higher rates of underlying health conditions lack access to testing due to limited resources and hospital closures as a result of the pandemic [4] . Military populations that operate in austere environments far removed from health care facilities also need access to adaptable molecular diagnostic approaches that enable early detection of SARS-CoV-2 or other highly contagious pathogens [5] . A cost-effective, easy-to-use application could address the capability gap faced by these communities.

The Biomeme Franklin TM three9 Real-Time PCR Thermocycler (Philadelphia, PA) system is a lightweight, portable, battery-powered qPCR device that can test biological samples without centrifugation, the use of frozen reagents, or a power source. Furthermore, the device is capable of multiplex detection of up to three targets in each sample, where nine samples can be tested in a single run. Real-time PCR results are displayed on a smartphone that is connected to the device via Bluetooth or a USB cable. The Biomeme SARS-CoV-2 Go-Strip assay is a qualitative test for the detection of SARS-CoV-2 viral RNA in nasopharyngeal, nasal, and oropharyngeal swab specimens, and nasopharyngeal washes or aspirates.

Biomeme received Emergency Use Authorization for their SARS-CoV-2 Go-Strip assay from the FDA on August 11, 2020 [6] .

The objective of this study was to evaluate the sensitivity and specificity of the Biomeme SARS-CoV-2

Go-Strips assay tested on the Biomeme Franklin TM three9 Real-Time PCR Thermocycler. We assessed the performance of the Biomeme SARS-CoV-2 Go-Strips assay utilizing Biomeme's M1 extraction method at a range of viral concentrations and tested several upper respiratory near-neighbor pathogens in the SARS-CoV-2 Go-Strips to evaluate cross-reactivity. Finally, we evaluated sensitivity and specificity of the Biomeme SARS-CoV-2 Go-Strips by testing clinical upper respiratory samples using the BioFire® Respiratory Panel 2.1 (RP2.1; Salt Lake City, UT) as a reference standard and validated results from the Biomeme SARS-CoV-2 Go-Strips using Biomeme SARS-CoV-2 Go-Plates tested on the ThermoFisher Quantstudio TM 7 Flex Real-Time PCR System (Waltham, MA).

RNA was extracted and purified using Biomeme's M1 Sample Prep Cartridge, a filtration-based manual extraction method where nucleic acid binds to a silica membrane inside of a piercing tool attached to a syringe. Sample is pumped through the membrane along the sealed cartridge chambers which contain lysis buffer, wash buffers, and an elution buffer. To evaluate the sensitivity of the Biomeme SARS-CoV-2 detection system, a dilution series was prepared. were prepared in RNase-free water from a stock concentration of 4.2 x 10 8 genome copies/mL. Additional concentrations of 2.10 x 10 3 and 1.05 x 10 3 copies/mL were prepared to assess performance near the threshold of detection. ATCC® VR-1986HK TM is a preparation of strain 2019-nCoV/USA-WA1/2020 inactivated by heating to 65°C of 30 minutes [7] . Each concentration was extracted and purified using the Biomeme M1 Sample Prep Cartridge Kit for RNA 2.0 according to manufacturer's instructions for use [8] . RNA from the M1 extraction process was tested in triplicate by adding 20 µL of RNA to the lyophilized master mix contained in each Go-Strip well. Amplification and detection of amplicon was performed in the Biomeme Franklin TM three9 Real-Time PCR device. Upon completion of RT-PCR, the Biomeme provides an interpreted result regarding which gene targets have been detected.

No analysis of amplification plots is required to interpret results. Sample results with neither ORF1ab nor S detection but with RPC amplification are considered negative for SARS-CoV-2. Samples results with only ORF1ab detection are considered presumptive positive for SARS-CoV-2. Sample results with S or both S and ORF1ab amplification are considered positive for SARS-COV-2. Sample results with no targets detected are considered invalid and need to be retested.

Nineteen near-neighbor upper respiratory viral and bacterial pathogens were tested on the Biomeme SARS-CoV-2 Go-Strips assay. Genomic material for each near-neighbor pathogen was purchased from ATCC ( Table 2) 

To validate the results on the Go-Strips, the same purified RNA was amplified and detected on Biomeme SARS-CoV-2 Go-Plates using the ThermoFisher Quantstudio TM 7 Flex. Biomeme SARS-CoV-2 Go-Plates contain the same reagents as the SARS-CoV-2 Go-Strips but in a 96-well format.

Statistical analyses were performed using R version 4.0.3 and the R packages 'epiR' and 'fmsb'. We used Cohen's kappa statistics to estimate agreement and test the null hypothesis that agreement was random (i.e. kappa statistic equals zero) [11] . We used McNemar's Chi-square test to test the null hypothesis that the platforms are equivalent in terms of sensitivity and specificity. We used probit regression to estimate the concentration for which 95% of tests would be positive for each of the two targets in the Biomeme Go-Strips. Confidence intervals for the 95% probit estimate were calculated using the inverse method in the 'investr' R package [12] .

Eight dilutions, ranging from 4.20 x 10 7 copies/mL to 420 copies/mL were tested to evaluate the performance of the Biomeme SARS-CoV-2 Go-Strip assay including M1 extraction at low viral concentrations (Table 1) 

We tested genomic material from nineteen near-neighbor upper respiratory bacterial and viral pathogens, including six strains of other coronaviruses, for cross-reactivity with targets in the Biomeme SARS-CoV-2 Go-Strips assay ( Table 2) . For each near-neighbor pathogen, no targets were detected in the SARS-CoV-2 Go-Strips. As a positive control, all near-neighbors upper respiratory pathogens that are target on the BioFire® RP2.1 were detected.

We Table 3 .

We validated our results on the Biomeme SARS-CoV-2 Go-Strips by analyzing the same clinical samples on the Biomeme SARS-CoV-2 Go-Plates. There was a high concordance between the Go-Strip and Go-Plate results. Cohen's kappa estimates ranged from 0.86 to 0.97 across specimen types. There were no statistically significant differences between the marginal frequencies, indicating that the Biomeme SARS-CoV-2 Go-Strips are equivalent in sensitivity to the Biomeme SARS-CoV-2 Go-Plates. Counts for the comparative testing, positive percent agreement, and negative percent agreement are shown in Table 4 .

As the Biomeme Franklin TM three9 Real-Time PCR device has been successfully deployed for military environmental surveillance applications [13] , we sought to evaluate if the system could potentially be used in remote settings for the detection of SARS-COV-2. This study demonstrates that the Biomeme Franklin TM three9 Real-Time PCR Thermocycler and SARS-CoV-2 Go-Strip Assay may be an effective platform for the detection of SARS-CoV-2 in remote settings. We showed that the assay can reliably detect SARS-CoV-2 at viral concentrations of 4.20 x 10 3 copies/mL and found no cross-reactivity with select near-neighbor upper respiratory tract pathogens. We also demonstrated agreement that is statistically significantly greater than chance between the Biomeme SARS-CoV-2 Go-Strips and BioFire® RP2.1 in all four clinical upper respiratory sample types tested including nasopharyngeal swab in VTM, the current gold standard sample used for RT-PCR detection of SARS-CoV-2. The Go-Strips showed reduced sensitivity as compared to the BioFire® RP2.1; however, these results were expected based on the differences in limit of detections reported by the manufacturers. BioFire® reports a limit of detection of 500 copies/mL for the RP2.1 and Biomeme reports a limit of detection of 1,800 copies/mL [8, 9] . A high number of false negatives occurred in oropharyngeal swabs and saliva which we suspect is due to differences in viral load at the various collection sites [14, 15] .

We speculate that the sensitivity of the Biomeme SARS-CoV-2 Go-Strip assay could be improved by higher. Testing in this study was done in a controlled laboratory environment so it is possible that performance of the Biomeme system would be negatively impacted in an austere environment. Further studies should be conducted to evaluate the Biomeme system in field conditions. Lastly, the sample collection in this study lagged identification of CLIA testing. In positive samples many participants may have cleared the virus or have substantially lower viral loads than they would upon an initial presentation to a clinical setting for testing earlier in the course of illness. As a result, we speculate that there is an enrichment for samples around or below the limit of detection for the devices in this study.

Thus, the positive percent agreement between the Biomeme SARS-CoV-2 Go-Strips and the BioFire® RP2.1 might be expected to be lower than if the study design compared clinic samples collected at the peak of viral shedding (earlier in the course of illness, closer to the typical presentation to the clinic for illness).

The Biomeme Franklin TM three9 and SARS-CoV-2 Go-Strip assay provide healthcare facilities, especially those in rural areas, and personnel in remote settings an affordable, easy-to-use molecular diagnostic platform that uses almost no space on the benchtop. The Biomeme system does not require specialized laboratory equipment or cold-storage reagents. Results for up to nine samples can be obtained in less than two hours, including the time for processing and extraction and minimal training is required to learn the procedure. Our evaluation of the sensitivity and specificity of the Biomeme SARS-CoV-2 Go-Strips supports results reported in the Food and Drug Administration to support Emergency Use Authorization. Future studies should be performed comparing the sensitivity and specificity of the Biomeme SAR-CoV-2 Go-Strips assay to other SARS-CoV-2 detection platforms to provide a better understanding of the performance of the system. Table 1 Dilution 

WHO Director-General's opening remarks at the media briefing on COVID-19 -11

Clinical evaluation of the BioFire(R) Respiratory Panel 2.1 and detection of SARS-CoV-2

The Unique Impact of COVID-19 on Older Adults in Rural Areas

SARS-CoV-2 in the U.S. Military -Lessons for Civil Society

SARS-CoV-2 Real-Time RT-PCR Test Emergency Use Authorization. 2020. p. Food and Drug Administration Emergency Use Authorization of Biomeme, Inc. SARS-CoV-2 Real-Time PCR Test

Heat-inactivated SARS-CoV-2, VR-1986HK™

Biomeme SARS-CoV-2 Real-Time RT-PCR Test. 2020. p. Biomeme SARS-CoV-2 Real-Time RT-PCR Test Instructions For Use

BioFire® Respiratory Panel 2.1 (RP2.1). 2020. p. BioFire® Respiratory Panel 2.1 (RP2.1) Instructions for Use

Permits Marketing of First SARS-CoV-2 Diagnostic Test Using Traditional Premarket Review Process. 2021. p. BioFire Respiratory Panel 2.1 is the First COVID-19 Diagnostic Test Granted Marketing Authorization Using the De Novo Review Pathway

The measurement of observer agreement for categorical data

An R Package for Inverse Estimation

Development of Assay Panel for Vector-Borne Disease Research Cooperative Agreement D17AC00026. 59th Medical Wing, Office of the Chief Scientist

Viral Load in the Saliva

Nasopharyngeal Swabs Are More Sensitive Than Oropharyngeal Swabs for COVID-19 Diagnosis and Monitoring the SARS-CoV-2 Load

SARS-CoV-2: Comparative analysis of different RNA extraction methods

Franklin™ three9 ISP Real-Time PCR Thermocycler (EAU). 2020. p. Franklin™ three9 ISP Real-Time PCR Thermocycler Emergency Use Authorization Instructions for Use

 Table 3 Clinical Evaluation of Biomeme SARS-CoV-2 Go-Strips with BioFire® RP2. 1 

☒ The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.☐The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: