key: cord-0950852-o9xa81r0
authors: La, My-Van; Koo, Seok Hwee; Jiang, Boran; Heng, Ying Xuan; Tan, Thean Yen
title: A Study of Analytical and Clinical Sensitivity of Aptima SARS-CoV-2 Assay (Hologic) and Proposals of Complementary Tests for SARS-CoV-2 Detection in Low Viral Load Specimens
date: 2021-12-14
journal: Curr Microbiol
DOI: 10.1007/s00284-021-02730-3
sha: 6daa3be23b45d5e276d40311fbede9a10eab7d72
doc_id: 950852
cord_uid: o9xa81r0

Early and accurate detection of SARS-CoV-2 is important for diagnosis and transmission control. The use of high-throughput and automated testing allows laboratories to better deliver diagnostic testing given manpower and resource limitations. We validated the clinical and analytical performance of the Hologic Panther Aptima SARS-CoV-2 assay with an emphasis on detection of specimens with low viral loads. The clinical performance was evaluated using 245 clinical specimens, against a comparator PCR-based laboratory developed test (LDT). The analytical performance was determined by replicate testing of contrived samples in a ten-fold dilution series (C(T) values 32–42, based on LDT). The Aptima assay had 96.7% overall percent agreement, 100% negative percent agreement and 88.1% positive percent agreement. It was able to consistently detect SARS-CoV-2 in contrived samples with C(T) = 32 by LDT (calculated 2354 copies/mL). The 95% limit of detection of the Aptima assay was estimated to be at LDT C(T) = 33 (equivalent to 870 copies/mL). The relative light units (RLU) × 1000 for 52 true positive clinical specimens was 962.2 ± 181.5, and that for the 186 true negative specimens was 264.6 ± 14.3. The Aptima assay was a reliable method with a high overall percent agreement against our comparator LDT. We propose that samples reported as negative by the Aptima assay with RLU > 350 be tested by a secondary method, in order to improve detection of samples with very low viral loads. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s00284-021-02730-3.

Since the first reported detection of SARS-CoV-2 in December 2019, SARS-CoV-2 has rapidly spread across the world as a global pandemic. At the time of writing in July 2021, the number of global cases has reached 192,284,207, with 4,136,518 reported deaths [1] .

Reliable and sensitive laboratory detection methods for SARS-CoV-2 play a critical role in surveillance and controlling viral transmission [2] . The current gold standard and the vast majority of existing test methods in use are nucleic acid amplification tests [2] , which are predominantly based on real-time reverse transcription-polymerase chain reaction (qRT-PCR). The Aptima SARS-CoV-2 assay (Hologic, Panther Systems) uses target capture, transcription-mediated amplification (TMA), and dual kinetic assay technologies for the amplification and detection of the target virus. The assay targets two conserved regions of the open reading frame (ORF1ab) region of the SARS-CoV-2 viral genome and is run on the Panther Instrument [3] . It received the United States Food and Drug Administration (US FDA) Emergency Use Authorization on May 14, 2020 [4] .

While there has been a recent emergence of comparison of different molecular testing methods for the virus [2, [5] [6] [7] [8] [9] [10] [11] [12] [13] , the clinical performance of the Aptima assay has not been extensively-characterized. In this study, we aimed to validate its performance against standard qRT-PCR for the detection of SARS-CoV-2 in clinical specimens, with an emphasis on test performance in samples with low viral loads.

Nasopharyngeal flocked swabs (n = 195) were obtained from persons with clinical symptoms of respiratory tract infection, from screening of asymptomatic individuals in high-risk groups in the community and migrant workers residing in dormitories. Swabs were placed into 3 mL universal transport medium (UTM) and maintained in a chilled cooler box before and during shipping to the testing site (Changi General Hospital, Singapore) on the same day of collection. Samples were processed within 16 h of sample receipt. The primary test method was a laboratory-developed test (LDT) with confirmatory testing of low-viral load positive samples by a second PCR assay. Samples were concurrently tested on the Hologic Panther system. In order to further establish performance for samples with a low viral load, an additional 50 retrospective UTM samples (nasopharyngeal, n = 2; nasopharyngeal and oropharyngeal, n = 13; nose and throat, n = 34; throat, n = 1) archived at − 70 °C were tested simultaneously on the Hologic Panther System [3] and the LDT. All experiments were run without replicates to be reflective of the actual diagnostic practice.

Nucleic acids were extracted from 150 μL of UTM using the Viral DNA/RNA isolation kit on the EX3600 automated nucleic acid extraction system (LifeRiver™, China), according to manufacturer's instructions. The primary LDT qRT-PCR assay was designed to detect the N-gene (SARS-CoV-2 nucleocapsid protein) with an internal control (bacteriophage MS2). The fragments were amplified in a total volume of 30 μL containing 1 X reaction mix (SuperScript™ III Platinum One Step qRT-PCR system, Invitrogen, Life Technologies, USA), 0.4 μM of each N-gene primer, 0.16 μM of N-gene probe, 0.125 μM of each MS2 primer, 0.125 μM of MS2 probe (Integrated DNA Technologies, Singapore) (Supplementary Table 1 ), 1 μL of enzyme mix, and 10 μL of RNA extract. The PCR conditions were cDNA synthesis at 55 °C for 10 min, predenaturation at 95 °C for 3 min, followed by 45 cycles of denaturation at 95 °C for 15 s, annealing at 58 °C for 30 s. The reactions were run on Rotor-Gene Q (Qiagen, Germany) with positive and non-template controls, and fluorescence signal acquired on the yellow and crimson channels for target and internal control assays, respectively. A cycle threshold (C T ) value < 32 for the N-gene was interpreted as a positive signal for the target, while newly diagnosed positive samples with C T ≥ 32 for the N-gene were confirmed by a second test method: either a second LDT qRT-PCR targeting the ORF1ab gene [14] or Xpert® Xpress SARS-CoV-2 (GeneXpert, USA) which targets both the E and N genes as per manufacturer's instructions. This is to rule out any false positives defined by the reference method-N-gene LDT (±secondary confirmatory testing).

Briefly, for the secondary LDT qRT-PCR assay, the fragments were amplified in a total volume of 25 μL containing 1 X master mix (QuantiTect Probe RT-PCR Kit, Qiagen, Germany), 0.4 μM of each ORF primer, 0.20 μM of ORF probe (Integrated DNA Technologies, Singapore) (Supplementary Table 1 ), 0.25 μL of RT mix, and 5μL of RNA extract. The PCR conditions were cDNA synthesis at 50 °C for 30 min, pre-denaturation at 95 °C for 15 min, followed by 45 cycles of denaturation at 94 °C for 15 s, annealing at 55 °C for 45 s. The reactions were run on Rotor-Gene Q (Qiagen, Germany) with positive and non-template controls, and fluorescence signal acquired on the green channel for the target. A C T value ≤ 35 for the ORF1ab gene was interpreted as a positive signal for the target. When performed as a second test method of confirmatory testing, a C T value > 35 was also interpreted as a positive signal for SARS-CoV-2.

A 1000 μL aliquot of the UTM specimen was transferred into an Aptima Multitest Tube containing 2.9 mL Specimen Transport Medium and then loaded onto the Panther Instrument and analysed per manufacturer's instructions. In brief, an aliquot (360 μL) was sampled from each reaction and processed with the Target Capture Reagent (containing an internal control). The purified nucleic acid was then used as the template for TMA reactions. The hybridisation of chemiluminescent probes to the amplicons emits light measured by a luminometer in relative light units (RLUs). Assay results were interpreted based upon the manufacturer's determined total RLU cut-off and reported as 'positive', 'negative' or 'invalid'. Samples with discordant results from LDT underwent repeated testing on Aptima assay but only the results from the first run were used in the analysis.

In order to determine the analytical sensitivity of the Aptima assay, contrived samples of C T = 32, was prepared by diluting a strongly positive UTM specimen containing the SARS-CoV-2 virus. From the contrived samples of C T = 32, further dilutions were performed to obtain contrived samples of assumed C T = 34, 35, 36, 38, 40 and 42. The copy numbers were derived from a separate experiment involving the running of a standard curve of serially-diluted synthesized positive plasmid control containing a portion of the SARS-CoV-2 nucleocapsid gene. Twenty replicates each for samples with C T = 32, 34 and 35, and only one reaction for each of the remaining C T values were run on the Panther Instrument. The RLU values were used in the analysis and compared with the C T values to determine the limit of detection (LOD).

Sample size was estimated according to observer agreement study design with Cohen's kappa (κ) method. We postulated that the discordance between "Aptima assay" and "reference method" was 0.2 with strong agreement (κ = 0.8) and width of confidence interval set at 0.2; a minimum of 222 samples were required [15] .

Overall percent agreement (OPA), positive percent agreement (PPA), and negative percent agreement (NPA) were performed with the N-gene LDT (±secondary confirmatory testing) serving as the reference method. The 95% LOD of Aptima assay was estimated using Probit regression. All analyses were performed using SPSS v20.0 (IBM Corp. Armonk, NY).

For all the tested 245 samples (prospective, 195; retrospective, 50), OPA was 97.1% (238/245), with 100% NPA (186/186) and 88.1% PPA (52/59). Among the 59 positive samples (prospective, 9; retrospective, 50) detected by the LDT, four (6.8%) were strong positives (C T < 24), 27 (45.8%) were moderate positives (24 ≤ C T < 32), and 28 (47.5%) (C T ≥ 32) were weak positives. For positive samples with C T < 32 as tested by the LDT (n = 31), there was 100% PPA. For positive samples with C T ≥ 32 (n = 28), seven samples were reported as negative on the first run of the Aptima assay, with four being reported as positive upon repeat testing. The Cohen's kappa value was 0.919 (Z = 14.43, P < 0.001), indicating almost perfect overall agreement [16] (Table 1) . Among the seven false negative samples, four were retrospective specimens and three were prospective specimens. Upon repeat testing on Aptima assay, three retrospective specimens and one prospective specimen turned positive.

The RLU (× 1000) [mean ± SD (min-max range)] for 52 true positive clinical specimens was 962.2 ± 181.5 (587-1133), and that for the 186 true negative specimens was 264.6 ± 14.3 (239-415). The RLU (× 1000) [mean ± SD (min-max range)] for seven false negative clinical specimens was 462.1 ± 92.1 (261-541). Correspondingly, they were also only weakly positive as defined by their C T values (34.1 ± 0.7). Figure 1 shows the boxplots of the distribution of the RLU values for the positive and negative clinical specimens defined by Aptima assay. Table 2 summarizes the  results of the clinical validation and Table 3 shows the C T and RLU values of the nine positive prospective specimens.

The results of the contrived samples (C T = 32-42) and the profile of their RLU values are shown in Table 4 . The copy numbers of contrived samples with C T = 32 were quantitated to be 2354 copies/mL. For each of the 20 replicates tested (C T values = 32, 34 and 35), the detection rate by the Aptima assay was 100%, 85%, and 75%, respectively. The 95% LOD of the Aptima assay was estimated to be 870 copies/mL (LDT-equivalent of C T = 33). All of the single samples with LDT C T ≥ 36 were not detected by the Aptima assay.

The RLU (× 1000) [mean ± SD (range)] among the replicates of the contrived samples (negative by the Aptima 

This present study compared the clinical sensitivity and specificity, and analytical sensitivity of the Aptima assay against the LDT as the reference gold standard. The results showed almost perfect agreement (Cohen's kappa of 0.919), excellent NPA (100%) and acceptable PPA (88.1%) for all the clinical specimens tested. While the assay was highly specific, it might miss detecting samples with low viral load. All the four strong positives (C T < 24) were symptomatic of respiratory tract infections. The vast majority (92.9%) of the 28 weak positives (C T ≥ 32) were accounted by surveillance swabs of confirmed cases whose viral loads might be reducing over time (n = 16) and asymptomatic individuals from the screening of migrant workers in dormitories (n = 10). Smith and colleagues evaluated the Aptima assay against two qRT-PCR comparators (Hologic Panther Fusion and BioFire Defense) using 150 retrospective and prospective specimens. The Aptima showed 100% NPA and 94.7% PPA, with a non-significant trend towards slightly lower sensitivity compared with the other two assays [2] . Mostafa et al. compared the analytical sensitivity of the Aptima assay against seven other commercial assays, and concluded that the Aptima assay could detect 100% of replicates at nucleocapsid concentration of ~ 1200-1400 copies/mL, with three comparator systems detecting all replicates at one log 10 lower concentration. However, in this comparison, only contrived replicates were tested and the number of replicates tested was not equivalent for all of the commercial systems [5] . Schneider et al. reported 95.7% PPA and 100% NPA of the Aptima assay against real-time RT-PCR [17] .

In our study, analytical validation shows that the Aptima assay was able to consistently detect SARS-CoV-2 in contrived samples up to C T = 32 (2354 copies/mL) (100%), beyond which the sensitivity dropped to 85% and 75% for samples with C T = 34 (589 copies/mL) and C T = 35 (294 copies/mL), respectively. The range of RLU (× 1000) values of contrived samples with C T = 32 (810-1140) were comparable with the clinical specimens with C T = 32.0 ± 0.1 (633-1024). All single samples run for C T ≥ 36 turned out negative by the Aptima assay. The 95% LOD (870 copies/mL) is within the range reported in published studies (500-1000 copies/mL) [2] . The LOD of the assay per manufacture's package was 600 NDU/mL for FDA SARS-CoV-2 Reference Panel Testing and 83 copies/mL for the test sample [3] .

It is of note that there are numerous real-time PCR assays (commercial and LDT) available for use around the world. The C T values of these different assays are not directly comparable since they do not use identical primers/probes, master mix, and amplification conditions and instruments. The Infectious Diseases Society of America and Association for Molecular Pathology joint statement (March 12, 2021) advised against the use of published SARS-CoV-2 PCR C T values for correlating with disease severity and transmissibility and for clinical decision-making [18] . Therefore, it is imperative to highlight that the C T values presented in this report cannot be extrapolated to the real-time PCR testing platforms in other laboratories or be applied for clinical decision-making purposes.

The RLU values generated by the Aptima assay were analysed in relation to the assay's ability to detect targets of varying viral load. The RLU (× 1000) values of true positive samples (587-1133) were significantly higher from those of true negative samples (239-415), with no values overlapping between the two datasets. This indicates that the assay could differentiate clearly between positive and negative samples. The RLU (× 1000) values of the seven false-negative clinical specimens ranged from 261 to 541, with only one sample with RLU (× 1000) of <300. For the analytical replicates, the RLU (× 1000) values of the undetected replicates with C T ≤ 35 ranged from 392 to 559. For detected replicates with C T ≤ 35, the lowest RLU (× 1000) was 564. Based on the data from the study, samples which are tested by the Aptima 265-1140 assay with RLU values that fall between 350 and 563 may be tested by an alternative method to potentially improve the detection sensitivity of this assay. In a low-prevalence setting, this approach would better leverage the high-throughput and automation of the Hologic Panther platform, while minimising the risk of missing positive samples with a low viral load. The adoption of this approach would successfully detect six out of the seven false negatives, giving rise to a false negative rate of 0.4% (1/245) among all samples tested. This would not translate to a sizable number of undetected cases in a low-prevalent environment. The Aptima assay is a high-throughput, automated test which reduces the requirements for trained laboratory personnel in molecular biology to perform the testing. Other advantages include the closed tube system which reduces the risk of extrinsic contamination, as well as the ability to continually load samples during testing, allowing results to be generated in 3.5 h. These advantages need to be balanced against a potential loss of sensitivity for the detection of samples with a very low viral load.

We recognize the limitations associated with the study. Firstly, this is a single-site study based on relatively small sample size (n = 245). However, this sample set comprised specimens of varying viral load from individuals across demographic profiles, which is reflective of the real-world situation. Secondly, the majority of the positive cases (50/59) were pre-selected by our current standard method; hence the clinical sensitivity of the Aptima assay might be an overestimate due to the potential omission of weakly positive samples (called as negative by the LDT ± secondary confirmatory assay). Based on the data from the contrived samples, we expect this bias to be minimal from the ascertained LOD of the Aptima assay. Thirdly, this study reports the performance of the assay up till the time of testing, which may vary depending on the circulating variants, including newly emerging strains of SARS-CoV-2 and their prevalence, which change over time. Lastly, the Hologic Panther system is a floor-standing instrument which may not be a feasible option for small-scale laboratories with space constraints.

To the best of our knowledge, this is the first report offering insights on the interpretation of the RLU values of the Aptima assay and its performance in samples with low viral loads. The Aptima assay was a reliable assay with a high overall percent agreement against our comparator LDT. We propose an additional step for laboratories which use the Aptima assay (samples reported as negative with RLU > 350 be tested by a secondary method), which may combine the benefits of automated testing and also potentially improve the detection of SARS-CoV-2.

The online version contains supplementary material available at https:// doi. org/ 10. 1007/ s00284-021-02730-3.

World Health Organization Coronavirus Disease (COVID-19) Dashboard

Analytical and clinical comparison of three nucleic acid amplification tests for SARS-CoV-2 detection

CoV-2 assay (Panther® System) product insert

Authorization of emergency use of certain medical devices during COVID-19; availability

Comparison of the analytical sensitivity of seven commonly used commercial SARS-CoV-2 automated molecular assays

High-throughput transcriptionmediated amplification on the hologic panther is a highly sensitive method of detection for SARS-CoV-2

Evaluation of the Aptima™ transcription-mediated amplification assay (Hologic®) for detecting SARS-CoV-2 in clinical specimens

Performance characteristics of a high-throughput automated transcription-mediated amplification test for SARS-CoV-2 detection

Comparison of commercially available and laboratory-developed assays for in vitro detection of SARS-CoV-2 in clinical laboratories

Direct comparison of SARS-CoV-2 analytical limits of detection across seven molecular assays

Comparison of a pointof-care assay and a high-complexity assay for detection of SARS-CoV-2 RNA

Comparison of four molecular in vitro diagnostic assays for the detection of SARS-CoV-2 in nasopharyngeal specimens

Comparison of the panther fusion and a laboratory-developed test targeting the envelope gene for detection of SARS-CoV-2

Molecular diagnosis of a novel coronavirus (2019-nCoV) causing an outbreak of pneumonia

Observer agreement studies

Interrater reliability: the kappa statistic

Evaluation of the analytical performance and specificity of a SARS-CoV-2 transcription-mediated amplification assay

Infectious Diseases Society of America and Association for Molecular Pathology joint statement on the use of SARS-CoV-2 PCR cycle threshold (Ct) values for clinical decision-making

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

The authors would like to thank biostatistician, Ms Pei Ting Tan for her contributions in the statistical analysis.

Ethical Approval In our local context, the development of diagnostic tests using existing anonymised samples for test validation purposes is an eligible category for ethics exemption. The SingHealth Centralized Institutional Review Board states that this category of studies does not require ethics review. This study was on the validation of the Hologic Aptima assay for use in routine COVID-19 diagnostic testing.

critically reviewed and edited the manuscript. MVL, BJ and YXH designed the methodology and performed the experiments. MVL and SHK analysed the study data and drafted the manuscript. All authors contributed to and approved the final manuscript.Funding This research did not receive any specific grant from funding agencies in the public, commercial, or non-profit sectors.Data Availability (Data Transparency) The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

The authors have no conflict of interest to declare that are relevant to the content of this article.

Consent to Participate Consent taking for the use of these anonymised samples is not required.