key: cord-0790872-rxx3v6cg
authors: Jahrsdörfer, Bernd; Kroschel, Joris; Ludwig, Carolin; Corman, Victor Max; Schwarz, Tatjana; Körper, Sixten; Rojewski, Markus; Lotfi, Ramin; Weinstock, Christof; Drosten, Christian; Seifried, Erhard; Stamminger, Thomas; Groß, Hans Jürgen; Schrezenmeier, Hubert
title: Independent side-by-side validation and comparison of four serological platforms for SARS-CoV-2 antibody testing
date: 2020-10-16
journal: J Infect Dis
DOI: 10.1093/infdis/jiaa656
sha: 20e250239789d884d62aa055ee619bf19cf7774f
doc_id: 790872
cord_uid: rxx3v6cg

Highly sensitive and specific platforms for the detection of anti-SARS-CoV-2 antibodies are becoming increasingly important for (1) evaluating potential SARS-CoV-2 convalescent plasma donors, (2) studying the spread of SARS-CoV-2 infections and (3) identifying individuals with seroconversion. This study provides a comparative validation of four anti-SARS-CoV-2 platforms. Unique feature of this study is the use of a representative cohort of COVID-19-convalescent patients with mild-to-moderate disease course. All platforms showed significant correlations with a SARS-CoV-2 plaque-reduction-neutralization test, with highest sensitivities for the Euroimmun and the Roche platforms, suggesting their preferential use for screening of persons at increased risk of SARS-CoV-2 infections.

In the context of the current pandemic with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the development and provisioning of serological test systems for the determination and observation of seroprevalence and seropositivity is crucial for both public health aspects as well as for specific protection of health care professionals [1] . Due to the initial lack of sufficient numbers of pharyngeal swab NAT tests, the number of confirmed SARS-CoV-2 infections in Germany and other countries has long been considered to be largely underestimated [2] . Nevertheless, a recent seroprevalence survey from Switzerland demonstrated that most of the population of Geneva remained uninfected until the end of the first SARS-CoV-2 wave [3] .

Interpretation of serological test results is complicated by the fact that the majority of the initial reports on COVID-19 naturally included hospitalized patients with more severe disease courses, associated with different seroconversion behavior and antibody titers than patients with asymptomatic to moderate COVID-19 courses [4, 5] . Caution must therefore be applied concerning the validation of commercial test platforms, since the companies´ availability of representative validation cohorts, particularly of positive control groups may have been limited or biased at the time tests were developed. In April 2020 we started screening patients who recovered from mild to moderate COVID-19 to evaluate their suitabililty as convalescent plasma donors within a prospective randomized clinical trial (CAPSID, 2020-001310-38, EudraCT No: 2020-001310-38, ClinicalTrials.gov-Identifier: NCT04433910). The cohort of these donors was found to be ideal to represent the general population including health care workers with regard to SARS-CoV-2 infection.

The present study is the first providing a side-by-side validation of four commercially available serological platforms (Euroimmun, Snibe/Medac, Roche and Abbott) for the detection of anti-SARS-CoV-2 antibodies by use of the above-mentioned COVID-19-convalescent cohort. Based on this cohort, we calculated assay performance indicators including sensititivity, specificity, positive and negative predictive values, and the concordances between the platforms. Moreover, our study correlates all serological results with a wild-type SARS-CoV-2 neutralisation assay, allowing predications on the predictive value of serology for a potential therapeutic efficacy of immune plasma from COVID-19-convalescent donors. 

Serum samples for the positive validation group (COVID-19 + ) were collected from individuals who presented to our institute for assessment as potential convalescent plasma donors for a planned Table 2 ). The highest analytical sensitivity with 97.5% was reached when the three analytes from the Euroimmun ELISA platform were combined. Sensitivities of the single analytes from the Euroimmun platform were close to the values provided by the manufacturer (Supplementary Table 1 ). Sensitivity of the Roche platform was 95.0%, sensitivity of the Abbott platform 81.5%. To our surprise, the Snibe/Medac system failed to provide a sufficient sensitivity (60.3%) to reliably detect COVID-19 + individuals from our validation cohort.

Analytical specificity on the other hand was highest with the Roche and the Abbott platforms (100% each), followed by a specificity of 97.2% with the Snibe/Medac and 90.9% with the Euroimmun platforms. Of note, combining the anti-nucleocapsid IgG analyte from Roche with the anti-spike IgG analyte from Euroimmun enabled us to increase the combined test sensitivity to 96.6%, while keeping specificity at 100% (Table 1B) .

When correlating the analytic results from the different platforms, strong correlations were found between the Euroimmun analytes anti-spike IgG, anti-spike IgA and anti-nucleocapsid IgG among each other (not shown), as well as with the analytes from the other three manufacturers (Figure 1A,   B ). Of note, the analytic results from the Roche platform for the COVID-19validation group form a very flat scatter-plot, which reflects our finding that more than 99% of the COI results from this group range between 0 and 0.2. This again illustrates the high specificity of the Roche platform.

A c c e p t e d M a n u s c r i p t 7 When considering individual concordances between the serologic platforms, Euroimmun and Roche stood out from the other platforms with an overall concordance rate of > 96% (Supplementary Table 3A ). Several samples from the COVID-19 + validation group were identified as positive by the Euroimmun, but not by the other platforms including the Roche and the Abbott platforms, although the COI values for some of these sample were very close to the cutoff values (Figure 1A, B, 

Particularly for therapeutic use of plasma from COVID-19-reconvalescent donors it is paramount to know in advance the potential of the harvested plasma products to inhibit SARS-CoV-2. Since viral plaque inhibition assays are laborious and time-consuming, serological platforms may allow a more rapid prediction of the potential neutralization capacity of a plasma product. We therefore correlated the results from all four serological platforms with the results from a wild-type SARS-CoV-2 neutralization assay [6, 7] . The strongest correlations were found for the three analytes from the Euroimmun platform ( Figure 1C) . In contrast, the weakest correlations were found with the analytes from Roche and Snibe/Medac, whereas the Abbott analyte showed an intermediate correlation ( Figure 1D) .

A c c e p t e d M a n u s c r i p t 8

Meanwhile several reports on SARS-CoV-2 antibody testing platforms have been published [8] [9] [10] .

The major results of these reports are confirmed by our present study. More importantly however, our study represents the first and most comprehensive side-by-side comparison of four independent and commercially available serological platforms in this regard. The need of serological SARS-CoV-2 antibody test methods for screening of the general population and for the establishment of so-called "immunity passports" is currently matter of debate in many countries, unfortunately very often strongly influenced by political and economical considerations. Independently of this discussion, health care professionals, who are naturally at a higher risk of being infected with SARS-CoV-2, will definitely benefit from serological tests with utmost sensitivity and specificity. In addition, the recruitment of donors for convalescent plasma requires screening tests to identify those candidates who mounted a strong humoral immune response.

Our comparative analysis demonstrated that the serological tests currently available on the market show striking differences regarding sensitivity, specificity, as well as positive and negative predictive values. Our data suggest that particularly the platforms from Euroimmun and from Roche provide excellent sensitivities, allowing the screening of health care professionals with frequent contact to COVID-19 patients. Besides, the Euroimmun platform appears to be particularly suitable to test potential convalescent plasma donors, since its analytes showed the strongest correlations with a wild-type SARS-CoV-2 neutralization assay. On the other hand, specificity was highest with the Roche and the Abbott platforms, suggesting both tests may be used for testing broader populations with a comparably low risk of having recently been infected with SARS-CoV-2. Of note, the widely used Snibe/Medac platform failed to detect a significant portion of COVID-19 patients with mild to moderate disease courses, so that our data rather discouraged its use for the above-mentioned indications in its current form. Importantly, we were recently informed by Snibe/Medac about efforts to improve their assays. A limited number of our validation samples was tested with novel kits from the company and indeed produced results that appeared to be much closer to the results from the other platforms. Nevertheless, due to limited time and material resources, the analysis of a higher number of samples with the optimized assays could not be performed.

A c c e p t e d M a n u s c r i p t 9

In summary, we present here an independent and comprehensive validation and comparative analysis of the serological anti-SARS-CoV-2 platforms from Euroimmun, Roche, Abbott and Snibe/Medac. The unique feature of our study is the use of a representative cohort of 119 COVID-19convalescent patients with a mild to moderate disease course, instead of hospitalized COVID-19

patients. Based on this cohort, the highest sensitivities were found for the Euroimmun and the Roche platforms, whereas the highest specificity was obtained with the Roche and the Abbott platforms. The Snibe/Medac platform had an extraordinarily low sensitivity in our COVID-19 + cohort, so that we excluded it from further use for the broad screening of health care staff and for our convalescent plasma donor program. Of note, a combination of the Euroimmun and the Roche platforms resulted in a combined test sensitivity of 96.6%, while keeping specificity at 100%. Based on these results, antibody screening of persons at increased risk of SARS-CoV-2 infections (e.g. health care professionals) can be performed with the Roche platform for initial testing and, in cases with increased risk or borderline COI values in the Roche test, the additional use of the Euroimmun platform. Moreover, the Euroimmun platform can be used for the screening of potential convalescent plasma donors (e.g. anti-sipke IgG, cutoff 1.1). In case of a reactive result, this screening can be followed by the PRNT assay as described. If PRNT50 OR PRNT90 titers are > 1:20, the donor may be accepted for convalescent plasma donor programs and may be invited for plasma donation. In case of non-reactivity in the initial screening with the Euroimmun assay, or in case of PRNT50 AND PRNT90 titers < 1:40 the donor appears rather unlikely to be suitable as convalescent plasma donor. Analytic results were obtained as OD ratios for the Euroimmune analytes, in AU/ml for the Snibe analytes, and as cutoff indices (COI) for both the Roche and the Abbott analytes. Note that the combination of the analytical results from the Roche (anti-nucleocapsid IgG) and the Euroimmun (anti-spike IgG) platforms resulted in improved overall assay performance indicators based on our validation cohorts (blue).

A c c e p t e d M a n u s c r i p t 15 Figure 1 

The COVID-19 pandemic: major risks to healthcare and other workers on the front line

Estimates of the severity of coronavirus disease 2019: a model-based analysis

Seroprevalence of anti-SARS-CoV-2 IgG antibodies in Geneva, Switzerland (SEROCoV-POP): a population-based study

Antibody responses to SARS-CoV-2 in patients with COVID-19

Clinical and immunological assessment of asymptomatic SARS-CoV-2 infections

Plaque assay for human coronavirus NL63 using human colon carcinoma cells

Transmission of MERS-coronavirus in household contacts

Evaluation of commercial and automated SARS-CoV-2 IgG and IgA ELISAs using coronavirus disease (COVID-19) patient samples

SARS-CoV-2 Antibody Testing -Questions to be asked

Evaluation of the EUROIMMUN Anti-SARS-CoV-2 ELISA Assay for detection of IgA and IgG antibodies

We thank Dr. Dorit Fabricius for careful and critical review of the manuscript. We also thank Marie Luisa Schmitt, Anja Richter, Marlies Just, Nicolas Heinemann, Brigitte Korte and Thomas Becker for technical assistance.The clinical trial "CAPSID" is supported by the German Ministry of Health (Bundesministerium für Gesundheit)