key: cord-0688819-8jatj3qd authors: Nilles, Eric J; Karlson, Elizabeth W; Norman, Maia; Gilboa, Tal; Fischinger, Stephanie; Atyeo, Caroline; Zhou, Guohai; Bennett, Christopher L; Tolan, Nicole V; Oganezova, Karina; Walt, David R; Alter, Galit; Simmons, Daimon P; Schur, Peter; Jarolim, Petr; Woolley, Ann; Baden, Lindsey R title: Evaluation of three commercial and two non-commercial immunoassays for the detection of prior infection to SARS-CoV-2 date: 2021-07-01 journal: J Appl Lab Med DOI: 10.1093/jalm/jfab072 sha: adf02b87d7e83395d4cc70262c432b5bedfa23b4 doc_id: 688819 cord_uid: 8jatj3qd BACKGROUND: Serological testing provides a record of prior infection with SARS-CoV-2, but assay performance requires independent assessment. METHODS: We evaluated three commercial (Roche Diagnostics pan-IG, Epitope Diagnostics IgM and IgG) and two non-commercial (Simoa, Ragon/MGH IgG) immunoassays against 1083 unique samples that included 251 PCR positive and 832 pre-pandemic samples. RESULTS: The Roche assay registered the highest specificity 99.6% (3/832 false positives), the Ragon/MGH assay 99.5% (4/832), the primary Simoa assay model 99.0% (8/832), and the Epitope IgG and IgM 99.0 (8/830) and 99.5% (4/830) respectively. Overall sensitivities for the Simoa, Roche pan-IG, Epitope IgG, Ragon/MGH IgG, and Epitope IgM were 92.0%, 82.9%, 82.5%, 64.5% and 47.0% respectively. The Simoa immunoassay demonstrated the highest sensitivity among samples stratified by days post symptom onset (PSO), < 8 days PSO (57.69%) 8-14 days PSO (93.51%), 15-21 days PSO (100%), and > 21 days PSO (95.18%). CONCLUSIONS: All assays demonstrated high to very high specificities while sensitivities were variable across assays. 46 Its important to patients and public health experts to have accurate antibody tests for detection 47 of prior COVID-19 infection, but the tests can have variable results. This paper compares three 48 commercial and two non-commercial assays for COVID-19 antibodies to determine sensitivity 49 and specificity of each assay among 251 samples known to be PCR test positive and 832 50 samples collected prior to the COVID-19 pandemic. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 111 Hospital. Commercial assays were performed according to manufacturer specifications. The 112 Simoa and Ragon/MGH assays were performed according to previously described methods. 9, 11 113 All samples were tested for investigatory purposes, not for clinical diagnostic testing. 114 Commercial assays but not non-commercial assays received Emergency Use Authorization from 115 the United States Food and Drug Administration and CE certification from the European 116 Medical Device Safety Service. 118 Threshold cutoffs for defining positive, negative or indeterminate/borderline test results were 119 defined according to manufacturer specifications for commercial assays. Threshold cutoffs and 120 result determination for the non-commercial assays were established by the respective 121 laboratories prior to the study according to methods previously described. 9-11 Given the Simoa 122 multiplex assay includes 12 output measures per sample (IgG, IgM, and IgA against four viral 123 epitopes), results were based upon three pre-study classification models-an "Early Model," 124 "Late Model" and full panel "12-Parameter Model." 10 The Early Model, which previously 125 demonstrated the best performance, includes four markers: IgA S1, IgA NC, IgG NC, and IgG 126 Spike. 10 127 Data analysis 128 We performed five primary independent analyses. One each for the Roche (pan-IG) and 155 infection, suggesting cross reactivity due to recent respiratory infections is unlikely to be an 156 important cause of false positives in these assays. However, no human common coronaviruses 157 (HCoV, e.g. 229E, NL63, OC43 or HKU1) were documented among these samples so these data 158 do not assess for HCoV-specific cross reactivity. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 199 sensitivity for samples collected ≥21 days PSO (89.2%) in this study than a recent large UK 200 performance study, that reported a sensitivity of 97.2% from 536 PCR-confirmed samples 201 collected ≥20 days PSO. 14 The sensitivity of the Epitope IgG assay was lower than package insert 202 data that report 100% for 30 PCR positive samples in the second week of disease and lower 203 than a German study that reported 100% sensitivity for 22 PCR positive samples 15 but similar to 204 a US-based study that reported sensitivities of 84% at 6-20 days and 91% at >20 days. 16 205 Surprisingly, assay sensitivities, particularly in the ≥21 day time period when almost all PCR-206 confirmed cases would be expected to have detectable antibodies, 17 were lower than expected 207 with only the Simoa assay maintaining high sensitivity (95.2%) consistent with a prior report. 208 Positive samples in this study were collected prior to COVID vaccine availability or the (known) 211 The strengths of this study is the large number of systematically collected and curated control 212 samples and the use of all samples across all assays, which is required to accurately compare 213 assays but is rare among head to head assay evaluations, 1-3 including COVID-19 samples 214 collected 1-14 days since symptom onset to assess early sensitivity, and using assays targeted 215 to a range of antigens (spike, receptor binding domain, nucleocapsid and S1). We also provide 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Table 4 . Assay sensitivities by days post-symptom onset USA) intended for the qualitative detection of pan-immunoglobulin antibodies against the 104 nucleocapsid (N) antigen; 6 EDI New Coronavirus COVID-19 enzyme-linked immunosorbent 105 assays (ELISA) (Epitope Diagnostics, USA) that detect IgG and IgM against the N antigen IgM and IgA against the receptor binding 107 domain (RBD); and the Single Molecule Array multiplex assay (Simoa) that detects IgG, IgM, and 108 IgA against the spike protein, S1 subunit, RBD, and NC. 9,10 The Ragon/MGH assay was 109 performed at the Ragon Institute of Massachusetts General Hospital, Massachusetts Institute of 110 Technology and Harvard; all other assays were performed at the Brigham and Women's 133 samples collected < 8, 8-14, 15-21, and >21 days post symptom onset (PSO) The Epitope IgM and Ragon/MGH (IgG) result, the specificity is lower (12/830 false positives 22 were from 700 pre-pandemic samples (3.1%) without recent 154 respiratory infection and 5 from 132 pre-pandemic samples (3.7%) with recent respiratory 177 We assessed the performance of three widely CoV-2 immunoassays using a panel of 251 PCR positive hospitalized cases, and 1083 well-179 characterized prepandemic samples. Assays targeted a range of antigens including spike, NC, 180 RBD and nad S1. Unlike most head to head SARS-CoV-2 immunoasssay performance When compared against other available data, the 197 sensitivity of the Roche pan-IG assay and Epitope assays were lower than manufacturer samples and electronic health record data Clinical Investigations, and Mass General Brigham Biobank, informatics and laboratory teams at Author Contributions: All authors confirmed they have contributed to the intellectual content of this 233 paper and have met the following 4 requirements: (a) significant contributions to the conception and 234 design, acquisition of data, or analysis and interpretation of data; (b) drafting or revising the article for 235 intellectual content; (c) final approval of the Karlson, statistical analysis, provision of study material or patients Tolan, administrative support, provision of study material or patients Walt is 246 an inventor of the Simoa technology, a founder of the company and also serves on its Board of 247 Directors. D.R. Walt's interests were reviewed and are managed by Brigham and Women's Hospital and 248 Mass General Roche Diagnostics Corporation Walt has a financial interest in Quanterix Corporation, a company that develops 251 an ultra-sensitive digital immunoassay platform Karlson is supported by 257 National Institutes of Health (U01 HG008685, OT2OD026553, P30 AR070253). D.P. Simmons is 307 four new commercial serologic assays for determination of SARS-CoV-2 IgG Test performance evaluation of SARS-CoV-2 310 serological assays. medRxiv Quantifying antibody kinetics and RNA shedding 313 during early-phase SARS-CoV-2 infection Differences of SARS-CoV-2 serological test performance 316 between hospitalized and outpatient COVID-19 cases