key: cord-0991534-518ycsp7
authors: Poljak, Mario; Valenčak, Anja Oštrbenk; Štamol, Tina; Seme, Katja
title: Head-to-head comparison of two rapid high-throughput automated electrochemiluminescence immunoassays targeting total antibodies to the SARS-CoV-2 nucleoprotein and spike protein receptor binding domain
date: 2021-03-05
journal: J Clin Virol
DOI: 10.1016/j.jcv.2021.104784
sha: 081414dbb171bfe0e050ff60827467bd4bc8c53e
doc_id: 991534
cord_uid: 518ycsp7

BACKGROUND: Accurate anti-SARS-CoV-2 assays are needed to inform diagnostic, therapeutic, and public health decisions. The first manufacturer-independent head-to-head comparison of two rapid high-throughput automated electrochemiluminescence double-antigen sandwich immunoassays targeting total anti-SARS-CoV-2 antibodies against two different viral proteins, Elecsys Anti-SARS-CoV-2 (Elecsys-N) and Elecsys Anti-SARS-CoV-2 S (Elecsys-S) (Roche Diagnostics), was performed in a routine setting during the exponential growth phase of the epidemic’s second wave. METHODS: The diagnostic specificity of Elecsys-N and Elecsys-S was initially evaluated on a panel of 572 pre-COVID-19 samples, showing 100% specificity of both assays. Elecsys-N/Elecsys-S head-to-head comparison used 3,416 consecutive blood samples from individuals that were tested for the presence of anti-SARS-CoV-2 within commercial out-of-pocket serologic testing. RESULTS: Elecsys-N/Elecsys-S head-to-head comparison showed overall agreement of 98.68% (3,371/3,416; 95% CI, 98.23–99.03%), positive agreement of 95.16% (884/929; 95% CI, 93.52–96.41%), and a high kappa value of 0.996 (95% CI, 0.956–0.976). Previous SARS-CoV-2 PCR positivity was identified in 14/24 (58.3%) Elecsys-N negative/Elecsys-S positive individuals and in 4/21 (19.0%) Elecsys-N positive/Elecsys-S negative individuals. CONCLUSION: The first Elecsys-N/Elecsys-S head-to-head comparison showed excellent agreement of two highly specific and rapid high-throughput automated anti-SARS-CoV-2 assays. An important question is whether laboratories offering two different antibody assays could benefit from combining the assays; if so, should use be concomitant or sequential—and, in the latter case, in which order? Based on our results, we favor concomitant over sequential Elecsys-N/Elecsys-S use when testing individuals for anti-SARS-CoV-2 antibodies in high-incidence settings; for example, during the exponential or stationary growth phase of the COVID-19 epidemic.

The availability of assays to detect antibodies against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) created excitement and hope among the laboratory community, government leaders, and the public [1] . Unfortunately, early in the pandemic, the global market was flooded with antibody assays of unproven performance and various governments purchased large quantities of ineffective tests [2, 3] . The situation improved with implementation of verification/authorization procedures and recommendations for antibody test utilization and result interpretation [1] [2] [3] [4] .

Antibody tests are a useful diagnostic aid, primarily for patients that present later in the disease course and are negative for SARS-CoV-2 RNA, when a lower-respiratory-tract sample cannot be collected, for diagnosing multisystem inflammatory syndrome in children, and to screen potential donors for convalescent-phase plasma therapy [5, 6] . Serologic testing may prove useful in determining immunity, stratifying individuals for vaccine receipt, and documenting vaccine response, which could inform return-to-work and travel decisions and other public health measures [5, 6] . Finally, they play an important role in understanding the epidemiology, including seroprevalence at the local, national, and global levels [5] [6] [7] .

Although several commercial anti-SARS-CoV-2 assays have received U.S. Food and Drug Administration (FDA) emergency-use authorization (EUA), most approved assays lack manufacturer-independent performance evaluations in peer-reviewed literature.

Here we present a manufacturer-independent head-to-head comparison of two rapid (18minute) high-throughput automated electrochemiluminescence double-antigen sandwich immunoassays targeting total anti-SARS-CoV-2 antibodies against two different viral proteins:

Elecsys Anti-SARS-CoV-2 (Elecsys-N) and Elecsys Anti-SARS-CoV-2 S (Elecsys-S) (Roche Diagnostics, Mannheim, Germany).

Elecsys-N is an assay for qualitative detection of total anti-SARS-CoV-2 antibodies against nucleoprotein (N) that received FDA EUA on May 3, 2020, and Conformitè Europëenne (CE) mark on April 28, 2020. The assay has been extensively evaluated by the manufacturer, showing 99.80% (95% confidence interval (CI), 99.69-99.88%) clinical specificity on 10,453 samples and 99.5% (95% CI, 97.0-100%) sensitivity on 185 samples obtained 14 days or later after SARS-CoV-2 PCR-confirmation. Elecsys-N has also been evaluated in several manufacturer-independent studies, with diagnostic specificity and sensitivity values spanning claims made by the manufacturer in most studies [1, 5, [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] , and it is consequently considered one of the most appropriate assays for seroprevalence surveys, especially in low-prevalence settings [6, 17] . This study evaluated Elecsys-N and Elecsys-S head-to-head in a routine setting during the exponential growth phase of the epidemic's second wave. During the 84-day study period, the cumulative number of PCR-confirmed COVID-19 cases in Slovenia increased 18.4-fold, from 6,105 to 112,048 (https://www.nijz.si/sl/dnevno-spremljanje-okuzb-s-sars-cov-2-covid-19), providing a challenging but informative environment for evaluating two highly specific anti-SARS-CoV-2 assays directed against different SARS-CoV-2 antigens.

Before head-to-head comparison, the diagnostic specificity of Elecsys-N and Elecsys-S was internally evaluated in May and September 2020, respectively, on a panel of 572 samples collected prior to the emergence of COVID-19 (Table 1) .

For head-to-head comparison, 3,416 consecutive blood samples received between October 1, 2020, and December 23, 2020 were tested in parallel using Elecsys-N and Elecsys-S on a cobas e411 analyzer following the manufacturer's instructions, using cut-off values for positive results of ≥1.0 and ≥0.8 U/mL, respectively. Blood samples were obtained from the same number of individuals that attended out-of-pocket anti-SARS-CoV-2 testing with a commercial test provider. In contrast to SARS-CoV-2 RNA testing [18] , which is fully covered by national health insurance, SARS-CoV-2 antibody testing in Slovenia is not reimbursed. Thus, the study population consisted of individuals that requested out-of-pocket anti-SARS-CoV-2 testing for several reasons: travel purposes; to check serological response after PCR-confirmed COVID-19 or clinically compatible but virologically non-confirmed COVID-19; recent contact with a person with COVID-19, but not eligible for PCR-testing; or pure curiosity.

Due to the inability of obtaining follow-up sample(s) from individuals with Elecsys-N/Elecsys-S discordant results, the national SARS-CoV-2 PCR notification database was consulted and all discrepant samples were additionally tested by two anti-SARS-CoV-2 antibody assays with excellent analytical performance proven in manufacturer-independent J o u r n a l P r e -p r o o f evaluations: SARS-CoV-2 Ab Elisa Kit (Wantai; Wantai Biological Pharmacy Enterprise Co, Beijing, China) detecting total antibodies against the S protein RBD [19] [20] [21] [22] and Architect SARS-CoV-2 IgG (Abbott; Abbott Diagnostics, IL, USA) detecting IgG antibodies against N protein [5, 8, 10, 13, 14] . Furthermore, 10% and 5% randomly selected samples with Elecsys-N/Elecsys-S concordantly positive and concordantly negative results, respectively, were additionally tested by Wantai and Abbott.

A contingency table was constructed to assess overall and positive agreements with 95%

CIs. The level of agreement between both tests was assessed using kappa statistics. All statistical analyses were performed using Excel (Microsoft, Redmond, WA, USA) and R software version 3.2.5 (Free Software Foundation, Boston, MA, USA).

Internal evaluation on the panel of 572 pre-COVID-19 samples showed 100% specificity of both assays ( Table 1 ). As shown in Table 2 

Accurate anti-SARS-CoV-2 assays are needed to inform diagnostic, therapeutic, and public health decisions [5, 23] . When selecting antibody assays, virologists must consider not only sensitivity and specificity, but also prevalence in the tested population, the intended use of results, sample throughput, test complexity, reagent and instrument availability, and cost per reportable result [5] . Especially assays' throughput and specificity are crucial parameters if J o u r n a l P r e -p r o o f large-scale antibody testing is desirable in a low-prevalence pre-vaccination environment [9, 23] .

This comparison showed high overall and positive agreement of two highly specific and rapid high-throughput automated assays. Equal distribution of Elecsys-N/Elecsys-S discordant results was observed. Such distribution of discordant results was confirmed by additional testing using two supplementary assays: Wantai detecting the equivalent total anti-S RBD antibodies as Elecsys-S and Abbott detecting IgG fraction of the total anti-N antibodies targeted by Elecsys-N. The recorded slight Elecsys-N/Abbott discordance is most probably a result of the presence of anti-N antibodies other than IgG detected by Elecsys-N and missed by Abbott.

Thus, although we were unable to obtain follow-up sample(s) from individuals with discordant results, we strongly believe that not more than 5% of discordant results are due to false positivity of one of the Elecsys assays. This is supported by: (i) extremely high specificity of both Elecsys assays recorded in the manufacturer's and manufacturer-independent evaluations An important open question is whether laboratories offering different antibody assays could benefit from combining the assays; if so, should use be concomitant or sequential-and, in the latter case, in which order? Previous studies showed that a two-assay algorithm improves the positive predictive value compared with an individual assay alone while maintaining the negative predictive value [5, 17, 27] . Thus, the two-assay approach was recently recommended for identifying potential convalescent-phase plasma donors and assessing candidacy for experimental COVID-19 therapeutics in PCR-negative patients with respiratory symptoms [5] .

As far as we know, the Elecsys-N and Elecsys-S manufacturer issued no recommendation for combination use, but the manufacturer's unpublished data showed that concomitant use of both assays could increase overall sensitivity (some convalescent patients were anti-S-only and some anti-N-only responders) and that sequential use (initially Elecsys-N followed by Elecsys-S for N-positives) could improve positive predictive value to 100% in low-prevalence settings. Based on our results, we favor concomitant over sequential Elecsys-N/Elecsys-S use when testing in 

Application, verification, and implementation of SARS-CoV-2 serologic assays with emergency use authorization

Antibody testing for coronavirus disease 2019: not ready for prime time

The complexities of SARS-CoV-2 serology

Infectious Diseases Society of America guidelines on the diagnosis of COVID-19: serologic testing

Evaluation of three commercial SARS-CoV-2 serologic assays and their performance in twotest algorithms

Diagnostic accuracy of serological tests and kinetics of severe acute respiratory syndrome coronavirus 2 antibody: a systematic review and meta-analysis

Comparison of SARS-CoV-2 serological tests with different antigen targets

Comparison of the clinical performances of the Abbott Alinity IgG, Abbott Architect IgM, and Roche Elecsys total SARS-CoV-2 antibody assays

Differences in performance characteristics among four high-throughput assays for the detection of antibodies against SARS-CoV-2 using a common set of patient samples

Performance characteristics of five immunoassays for SARS-CoV-2: a head-to-head benchmark comparison

Monitoring antibody response following SARS-CoV-2 infection: diagnostic efficiency of 4 automated immunoassays

Comparative clinical evaluation of the Roche Elecsys and Abbott Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) serology assays for Coronavirus Disease 2019 (COVID-19)

Head-to-head evaluation on diagnostic accuracies of six SARS-CoV-2 serological assays

Antibody response against SARS-CoV-2 spike protein and nucleoprotein evaluated by four automated immunoassays and three ELISAs

Variable performance in 6 commercial SARS-CoV-2 antibody assays may affect convalescent plasma and seroprevalence screening, In press Am

Longitudinal serological analysis and neutralizing antibody levels in coronavirus disease 2019 convalescent patients

Diagnostic accuracy of serological tests for covid-19: systematic review and meta-analysis

Clinical evaluation of the cobas SARS-CoV-2 Test and a diagnostic platform switch during 48 hours in the midst of the COVID-19 Pandemic

Humoral immune response to SARS-CoV-2 in Iceland

An evaluation of COVID-19 serological assays informs future diagnostics and exposure assessment

SARS-CoV-2 seroprevalence survey among 17,971 healthcare and administrative personnel at hospitals, pre-hospital services, and specialist practitioners in the Central Denmark Region

Comparison of sixteen serological SARS-CoV-2 immunoassays in sixteen clinical laboratories

Testing for SARS-CoV-2 antibodies

Antibody status and incidence of SARS-CoV-2 infection in health care workers

The duration, dynamics and determinants of SARS-CoV-2 antibody responses in individual healthcare workers

Antibody persistence in the first six months following SARS-CoV-2 infection among hospital workers: a prospective longitudinal study

We thank Robert Krošelj and Maja Accetto Kos for their excellent laboratory assistance.J o u r n a l P r e -p r o o f