key: cord-1014762-q5tgqb6w authors: Donato, Leslie J.; Theel, Elitza S.; Baumann, Nikola A.; Bridgeman, Amber R.; Blommel, Joseph H.; Wu, Yanhong; Karon, Brad S. title: Evaluation of the Genalyte Maverick SARS-CoV-2 Multi-Antigen Serology Panel date: 2021-06-19 journal: Journal of clinical virology plus DOI: 10.1016/j.jcvp.2021.100030 sha: 119acd7387da0bd21a8d01d84b32411b3e4f7f94 doc_id: 1014762 cord_uid: q5tgqb6w Serologic testing for SARS-CoV-2 can be used for evaluation of past infection in individual patients and for community seroprevalence studies. We evaluated the analytical and clinical performance of the Genalyte Maverick SARS-CoV-2 Multi-Antigen Serology Panel compared to the Roche Elecsys Anti-SARS-CoV-2 nucleocapsid (NC) qualitative immunoassay, using well characterized clinical serum samples. A total of 143 pre-pandemic sera and 48 sera collected from patients with a negative molecular SARS-CoV-2 result were used for specificity studies. For sensitivity analyses, 179 sera were used, obtained 3-7 days, 8-14 days, or ≥ 15 days after symptom onset from patients with confirmed SARS-CoV-2 infection. Specificity was determined to be 95.3% (182/191) for the Genalyte Maverick. Overall sensitivity of the Genalyte Maverick was similar to that observed for the Roche Elecsys NC test, 79.3% (142/179) vs. 76.5% (137/179), respectively. Genalyte Maverick trended, without statistical significance, towards higher sensitivity as compared to the Roche Elecsys NC test in the 3-7 days (11/25 vs. 9/25, respectively) and 8-14 days (21/28 vs. 19/28, respectively) post-symptom onset sample sets, but was identical in the ≥ 15 days post-symptom onset group (106/116 vs. 106/116, respectively). Therefore, the Genalyte Maverick serologic test had similar overall sensitivity to the Roche Elecsys NC assay, but may have slightly improved sensitivity for early seroconversion. The lower Genalyte Maverick specificity as compared to the Roche Elecsys NC assay as reported by other studies (>99%), may necessitate confirmatory testing of positive Genalyte Maverick results if implemented for clinical use. The global pandemic from severe acute respiratory syndrome-coronoavirus-2 (SARS-CoV-2) necessitated the development of multiple laboratory methods to assess both active and prior coronavirus disease-2019 . While molecular and antigen detection assays are used to identify active viral replication, serologic assays to detect the body's humoral immune response to SARS-CoV-2 are generally used to document previous infection. The clinical utility of SARS-CoV-2 serologic tests are limited, but include investigating local and community seroprevalence, assessing whether individual patients were previously infected, and identification of COVID-19 convalescent plasma donors (1) . While serologic testing can be utilized for determining immunity against other vaccine-preventable diseases (2, 3), a minimum antibody 'immunity threshold ' has not yet been established for SARS-CoV-2 and postvaccination serologic testing is not currently recommended. Depending on their design, serologic tests will detect IgM and/or IgG antibodies, with or without immunoglobulin differentiation, to SARS-CoV-2 in human blood. Seroconversion rates peak at 4-5 weeks post-symptom onset (4) . The SARS-CoV-2 target proteins used in serologic tests most commonly include recombinant, full or partial S (spike) or subunit 1/2 (S1, S2) proteins, S1 receptor binding domain (RBD), or NC (nucleocapsid) protein. Several methodologies for serologic testing have been developed including chemiluminescence immunoassays, enzyme-linked immunosorbent assays (ELISA), and lateral flow immunoassays for use in the central laboratory or at the point of care (5) (6) (7) . Currently, over 70 serology tests have received emergency use authorization (EUA) by the Food and Drug administration (FDA); 19 for high complexity testing only, 51 for high/moderate complexity testing, and 5 for high/moderate/waived complexity testing (https://www.fda.gov/medical-devices/coronavirusdisease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas, accessed 4/13/2021). The Genalyte Maverick SARS-CoV-2 Multi-Antigen Serology Panel v2 received FDA EUA for use in high or moderate complexity laboratories on October 8, 2020 and is a qualitative detection system for antibodies to SARS-CoV-2. The panel includes detection of IgG and IgM against five SARS-CoV-2 antigens (full length NC, full length S, S1, S2 and RBD), the SARS-CoV NC antigen, the Middle East respiratory syndrome (MERS) S1 antigen, and two influenza A hemagglutinin (H) antigens (H1 and H3). The test is unique among the available SARS-CoV-2 serology assays because it semi-quantitatively detects both IgM and IgG antibodies to multiple SARS-CoV-2 antigens and three other respiratory viruses. The system is based on photonic ring resonance technology on a silicon chip, and using a machine learning algorithm, simultaneously measures and interprets all reactions, releasing results within approximately 20 minutes. Here, we evaluated the test performance of the Genalyte SARS-CoV-2 serology panel against an automated reference method for detection of total antibodies against the SARS-CoV-2 NC antigen. The SARS-CoV-2 antibody negative sample set consisted of 143 pre-pandemic, residual sera collected in Rochester, MN between May 8, 2015 and November 30, 2019, and stored at -80 °C until thawed for the study. An additional 48 presumed SARS-CoV-2 antibody negative sera were residual specimens obtained primarily on May 1-3, 2020 from asymptomatic patients undergoing pre-procedural SARS-CoV-2 NAAT screening and antibody testing. Presumed negative sera were negative by the EuroImmun Anti-SARS-CoV-2 IgG ELISA (Lubeck, Germany), and collected from patients with a negative nucleic acid amplification test (NAAT) for SARS-CoV-2, with NAAT performed either on the day of blood collection or within the preceding 3 days. NAAT testing was performed in a CLIA-certified laboratory by one of several methods with FDA EUA. Presumed negative sera were stored at -80 °C until being thawed for the study. The SARS-CoV-2 antibody positive samples consisted of 179 residual sera from patients who had tested positive for SARS-CoV-2 by one of several FDA EUA NAAT tests. Serum samples were characterized as being collected 3-7 days, 8-14 days or, ≥ 15 days after symptom onset. For serum collected within 15 days of NAAT testing, electronic medical record review was performed to determine the number of days between symptom onset and blood collection. For serum collected more than 15 days after NAAT testing, no electronic medical record review was performed, and these samples were assumed to fall into the ≥15 days from symptom onset category. The range of days between NAAT and blood collection for these samples was 15 to 159 days, with >85% collected within 60 days of NAAT testing. Serum samples were initially stored up to 3 days on gel at 4-6 °C before being aliquoted and stored at 4-6 °C for an additional 4 days. The aliquots were then frozen and stored at -80 °C until thawed for the study. For these samples, the care environment (inpatient, outpatient, emergency department) at the time of blood collection was recorded. Among the 179 samples, 89 were collected from inpatients, 86 were collected from outpatients, and 4 were collected in the emergency department. Among the 116 samples collected from patients ≥15 days after a positive NAAT or onset of symptoms, 31 were collected from inpatients, 81 were collected from outpatients and 4 were collected in the emergency department. A second set of 40 SARS-CoV-2 antibody positive samples were characterized as positive by both the Roche Diagnostics Elecsys Anti-SARS-CoV-2 NC Total Antibody and the Ortho-Clinical Diagnostics VITROS Anti-SARS-CoV-2 IgG serologic tests. No NAAT test results or other patient clinical information were available for these samples. The samples were obtained from residual positive Ortho VITROS results and stored at -80 °C and thawed twice; once for testing on Roche total antibody assay and once for testing by the Genalyte assay. SARS-CoV-2 serologic assays: All serum samples were tested on a Genalyte Maverick instrument using the Maverick SARS-CoV-2 Multi-Antigen Serology panel v.2 assay (Genalyte Inc, San Diego CA). The assay uses photonic ring resonance (8) for the qualitative detection of total antibodies (IgG and IgM) to SARS-CoV-2 using EDTA whole blood, EDTA plasma, or serum samples. The assay uses a silicon chip to which 13 antigens are adhered, and the instrument detects changes in resonance wavelength as antibodies (IgG and/or IgM) are bound. The viral antigens used include 5 SARS-CoV-2 antigens (NC, S, S1, S2 and RBD), 4 antigens derived from commonly circulating coronaviruses (SARS-CoV-229E S, SARS-CoV-NL63 NC, SARS-CoV-OC43 S and SARS-CoV-HKU1 S), 2 influenza A hemagglutinin antigens (H1 and H3), and the NC and S1 antigens derived from SARS-CoV and MERS, respectively. External positive and negative quality controls were run daily, and serum samples were analyzed according to manufacturer's instructions for use, including assay calibration. Each chip has two channels allowing for two samples to be analyzed, and the instrument has two bays such that 4 samples can be analyzed simultaneously, with testing time of approximately 20 minutes. Test results are transmitted to the CloudLab software, where signals from the 26 antigens (separate IgG and IgM signals for the 13 antigens), along with additional control signals, are analyzed by the proprietary algorithm to determine whether each run (channel) was valid. If the run is valid, the algorithm returns results for total antibody of negative, indeterminate, or positive (EUA version of software). Invalid results that remained invalid upon retesting were excluded from our analysis. In the development version of the software used for this study, results returned from the CloudLab software included overall probability score (0-1), with a probability score of 0-0.45 interpreted as negative, 0.451-0.549 interpreted as indeterminate, and 0.55-1 interpreted as positive. The Roche Diagnostics Elecsys Anti-SARS-CoV-2 assay (Roche Diagnostics, Indianapolis IN) used in our automated central laboratory was used as the primary reference method against which the Genalyte assay was compared. The Roche Elecsys assay is immunoglobulin agnostic, uses a recombinant SARS-CoV-2 NC antigen, and is designed as a dual-antigen binding sandwich immunoassay. Testing was performed on a Roche Cobas 8000 e801immunoassay analyzer. Depending on timing of sample collection relative to symptom onset, the sensitivity of the Roche Elecsys assay ranges from approximately 80% to 95.8% in sera collected 2-3 or ≥4 weeks post-symptom onset, respectively, with specificity approaching 100% (9-11). The Ortho-Clinical Diagnostics Anti-SARS-CoV-2 IgG assay (Ortho Clinical Diagnostics, Rochester, NY) uses an immunometric two stage reaction to detect IgG antibodies against the S protein, with testing performed on the VITROS 3600 analyzer. The EuroImmun Anti-SARS-CoV-2 IgG ELISA (EuroImmun US Inc, Lubeck, Germany) uses microplate strips, each with 8 wells coated with recombinant SARS-CoV-2 S1 and was performed on the Dynex Agility instrument (Dynex Technologies, Chantilly VA). Assay performance characteristics for the Ortho-Clinical and EuroImmun assays have been described previously (12) . All assays have received FDA EUA and were performed according to manufacturer instructions, without deviation, in a CLIA-certified laboratory by trained laboratory technologists. Sensitivity: The Genalyte Maverick assay detected total antibody more frequently than the Roche Elecsys in the 179 sera from SARS-CoV-2 NAAT positive patients (Table 1) . Genalyte Maverick also detected antibody in more samples collected either 3-7 days or 8-14 days after symptom onset as compared to the Roche Elecsys. The higher sensitivity of the Genalyte assay as compared to the Roche test for early detection was seen in samples collected 5-10 days after symptom onset (Figure 1 ). Among 116 samples collected from patients with symptom onset ≥15 days or with NAAT testing ≥15 days before serum collection, the Genalyte Maverick and Roche Elecsys tests were both positive in 91.4% of samples (Table 1) . Among the 40 samples previously positive by both the Roche Elecsys and Ortho VITROS IgG assays, 39 of 40 (97.5%) were also positive by the Genalyte Maverick assay. Test performance is critically important when using SARS-CoV-2 serologic tests to detect disease seroprevalence in a community or prior individual infection. Given the variable disease prevalence, high specificity of serologic assays is essential in order to ensure accurate results. Currently, the recommended specificity for serologic assays, as defined by the Centers for Disease Control and Prevention (CDC) is ≥ 99.5% (https://www.cdc.gov/coronavirus/2019ncov/lab/resources/antibody-tests-guidelines.html , accessed March 18, 2021) . Serologic test sensitivity, while equally important, is expectedly more variable given the multiple factors that impact timing of seroconversion (e.g., disease severity, immune status, specimen collection timing, etc.). Here, we show the test performance of the Genalyte Maverick serologic test compared to the reference Roche Elecsys NC assay, using clinical samples that were well characterized for COVID-19 status and date of symptom onset. The Genalyte Maverick serologic test had similar, if not slightly improved, sensitivity for detection of antibodies to SARS-CoV-2 in confirmed samples as compared to the Roche Elecsys assay. For samples collected 15 or more days after symptom onset or NAAT testing, the sensitivity of the Genalyte and reference method were identical at 91.4%. However, the Genalyte assay showed slightly better sensitivity in samples collected both 3-7 days (44.0% vs. 36.0%) and 8-14 days (75.0% vs. 67.9%) after symptom onset. Although larger data sets are necessary to demonstrate whether these differences are statistically significant, it seems likely that use of multiple antigens in the Genalyte Maverick panel may allow for earlier detection of antibodies to SARS-CoV-2. The Genalyte Maverick assay uses five different target peptides for SARS-CoV-2 and the signals from all antigens are used to determine test positivity. In contrast, the Roche Elecsys assay relies on antibody detection against only the NC antigen. Sensitivity of the Genalyte assay from COVID-19 confirmed patients followed a similar trend relative to other assays, showing lower positivity (~40%) within the first week and increasing to near 100% in samples ≥15 days post symptom onset (5, 13, 14) . The 91% positivity rate of samples collected ≥15 days post symptom onset in our sample set is lower than reported in other studies. This lower positivity rate is unlikely an analytical performance issue given that the Genalyte and Roche tests reported identical positive rates in this cohort and nearly identical rates in another study (15) . It is more likely that our cohort included samples from patients with a lower rate of seroconversion because many (48% of samples overall and 70% of samples collected ≥15 days from symptom onset or NAAT testing) of these specimens were obtained from patients treated in the outpatient practice and likely experienced more mild disease. As has been reported, asymptomatic and mildly infected patients typically have lower anti-SARS-CoV-2 antibody levels and may be less likely to seroconvert compared to those with more severe symptoms (16) (17) (18) . For the purposes of prevalence studies, the false positive rate of any assay is critical when considering testing individuals in low-prevalence settings. Therefore, the specificity of a serologic test would ideally be as close to 100% as possible. In our study, the specificity of the Genalyte test was found to be approximately 95%. Similar to methods used in other studies, analytic specificity was determined in part using pre-pandemic samples collected well before the first COVID-19 case was identified in the United States, and in part, using samples collected from patients with negative SARS-CoV-2 NAAT tests. The Genalyte assay specificity was lower than that reported for the Roche Elecsys assay(~100%) (19) . The difference in specificity may be related to the antigens used in the two assays and the different analytical methods. The NC protein solely used by the Roche Elecsys assay is relatively small with a highly conserved sequence compared to the S protein which is larger and less well conserved. Thus, serologic tests that use both the NC and S proteins, such as the Genalyte test, may be more susceptible to crossreacting antibodies from other, closely related CoVs. The Genalyte panel compensates for this by measuring antibodies to both SARS-CoV-2 and commonly circulating CoV antigens and by using a machine learning algorithm to determine positivity for SARS-CoV-2 total antibody. While this approach appears to allow greater sensitivity for early detection of antibody to SARS-CoV-2, it may result in somewhat decreased specificity compared to the dual-antigen binding, automated immunoassay using nucleocapsid antigens. The specificity shown by Genalyte might be acceptable for some clinical situations, especially where the pre-test probability is high. Several limitations to our study should be noted. We included only residual samples and not fresh collections so analytical problems or assay inaccuracies from sample storage might have affected our results. Additionally, assay interferences due to fibrin strands or processing of fresh blood samples that might occur with use of fresh samples may have been missed. Although the number and characterization of samples used in the study was similar to that included in previous publications comparing SARS-CoV-2 assays (5, 9, 19) , the study size was not large enough to assess the statistical significance of differences in sensitivity and specificity between tests. Additional studies are necessary to determine whether the Genalyte method can consistently detect SARS-CoV-2 antibodies earlier than other, high-throughput, automated serologic assays. Lastly, it is unclear how the Genalyte test would perform in patients further removed from their initial COVID-19 infection, as few samples (<15%) were collected more than 60 days after symptom onset or date of NAAT testing. Overall, the Genalyte Maverick SARS-CoV-2 Multi-Antigen Serology Panel v2 showed good sensitivity for detection of antibodies to SARS-CoV-2, perhaps even better sensitivity than the reference method, in samples collected within the first two weeks of symptom onset. However, the modest decrease in specificity compared to the reference method may necessitate confirmatory testing if the test were to be utilized for population screening or for assessing previous infection in low prevalence populations. The Role of Antibody Testing for SARS-CoV-2: Is There One? Measles antibody: reevaluation of protective titers Rubella immunity. Defining the level of protective antibody Dynamics and Correlation Among Viral Positivity, Seroconversion, and Disease Severity in COVID-19 : A Retrospective Study Evaluation of Serological SARS-CoV-2 Lateral Flow Assays for Rapid Point-of-Care Testing Performance evaluation of antibody-based point-of-care devices intended for the identification of immune responses to SARS-CoV-2 Comparison of SARS-CoV-2 serological tests with different antigen targets Photonic ring resonance is a versatile platform for performing multiplex immunoassays in real time Head-to-Head Comparison of Two SARS-CoV-2 Serology Assays Comparison of the diagnostic sensitivity of SARS-CoV-2 nucleoprotein and glycoprotein-based antibody tests Evaluation of Six Commercial Mid-to High-Volume Antibody and Six Point-of-Care Lateral Flow Assays for Detection of SARS-CoV-2 Antibodies Performance Characteristics of Four High-Throughput Immunoassays for Detection of IgG Antibodies against SARS-CoV-2 Antibody Responses to SARS-CoV-2 in Patients With Novel Coronavirus Disease Target specific serologic analysis of COVID-19 convalescent plasma Clinical and immunological assessment of asymptomatic SARS-CoV-2 infections SARS-CoV-2-IgG response is different in COVID-19 outpatients and asymptomatic contact persons Disease severity dictates SARS-CoV-2-specific neutralizing antibody responses in COVID-19 Comparative Clinical Evaluation of the Roche Elecsys and Abbott Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Serology Assays for Coronavirus Disease 2019 (COVID-19) The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.