key: cord-0999808-7cy44u26
authors: Infantino, Maria; Pieri, Massimo; Nuccetelli, Marzia; Grossi, Valentina; Lari, Barbara; Tomassetti, Flaminia; Calugi, Graziella; Pancani, Silvia; Benucci, Maurizio; Casprini, Patrizia; Manfredi, Mariangela; Bernardini, Sergio
title: The who international standard for covid-19 serological tests: towards harmonization of anti-spike assays
date: 2021-08-30
journal: Int Immunopharmacol
DOI: 10.1016/j.intimp.2021.108095
sha: 0b22387cdd90301b94234b5a126f7e75e7ec9810
doc_id: 999808
cord_uid: 7cy44u26

BACKGROUND AND AIMS: SARS-CoV-2 antibody assays are relevant in managing the COVID-19 pandemic, providing valuable data on the immunization status of the population. However, current serology tests are highly variable, due to their different characteristics and to the lack of reference materials. The aim of the World Health Organization (WHO) first International Standard (IS) for anti-SARS-CoV-2 immunoglobulin is to harmonize humoral immune response assessment after natural infection or vaccination, and recommend reporting the results for binding activity in Binding Antibody Units (BAU). MATERIALS AND METHODS: This study analyzed six commercial quantitative anti-SARS-CoV-2 S-protein assays in a head-to-head comparison, using the manufacturers' conversion factors for the WHO IS to obtain BAU/mL values. RESULTS: Our data showed good alignment up to 1000 BAU/mL, then began to disperse, exhibiting some discrepancies. Moreover, correlations among methods varied with Cohen’s Kappa ranging from 0.580 to 1.00, with the lowest agreement values for kits using different target antigens or different antibody isotypes, making it clear that the laboratory report should include this information. Values expressed as BAU/ml showed a reduced between-assays variability compared to AU/ml (median coefficients of variation 0.38 and 0.68, respectively; p<0.001). CONCLUSION: On the basis of these data at present anti-SARS CoV-2 serological assays’ results are not interchangeable, and, more importantly, individual immune monitoring should be performed with the same method.

SARS-CoV-2 antibody assays have been and continue to be essential in managing the COVID-19 pandemic [1] [2] [3] .

Since the beginning, the US Food and Drug Administration (FDA) has issued Emergency Use Authorizations (EUA) for hundreds of serological assays to support COVID-19 diagnosis [4] .

As of 12 April 2021, there are at least 655 immunoassays for antibodies that are either commercially available or in development, according to the Foundation for Innovative New Diagnostics (FIND) (https://www.finddx.org/covid-19/pipeline/) [5] , a World Health Organization (WHO) collaborating center for Laboratory Strengthening and Diagnostic Technology Evaluation.

Simultaneously, in September 2020 the WHO issued the target product profiles for COVID-19 diagnostics, indicating 95-97% and 98-99% as acceptable criteria for sensitivity and specificity, respectively [6] .

Nowadays antibody testing may provide valuable data on: 1) surveillance and epidemiological assessment to evaluate the immunization status of the population [7] [8] ; 2) diagnosis of suspected cases with negative viral RNA test to increased diagnostic sensitivity for COVID-19 patients [9] [10] [11] ; 3) selection of hyperimmune plasma for convalescent plasma therapy [12] [13] ; 4) therapeutic antibody development and evaluation [14] [15] ; 5) assessment of COVID-19 vaccines immune responses and durability [16] [17] . Regarding vaccine application, Gudbjartsson et al. highlighted the utility of antibody assays as a highly cost-effective tool for a vaccination strategy to promote public health [18] . Although this new role has not yet been fully validated scientifically, as we do not know how long persisting antibodies confer protection, it is increasingly clear that antibody assays are needed to monitor and check responses to vaccines to support the interpretation of the clinical outcomes.

For this purpose, standardized anti-SARS-CoV-2 quantitative and neutralizing assays are pivotal in assessing immune responses to vaccines [19] , and it will become essential if a correlation of protection level could be identified with an antibody titre.

Among the serological assays, those targeting the spike protein, against the Receptor Binding Domain (RBD), the Subunit 1 (S1) or the full Spike (S), have played an important role in vaccine campaigns. However, current anti-S protein assays are highly variable in clinical practice [20] according to the different neutralizing activities, possibly due to a lack of reference materials. The Centers for Disease Control and Prevention (CDC) recently highlighted the need for standardized SARS-CoV-2 quantitative IgG and neutralization assays [21] . Improvements in antibody assay comparability are anticipated with the recent introduction of the WHO first International Standard (IS) for anti-SARS-CoV-2 immunoglobulin (NIBSC code 20/136), with the aim of harmonizing immune response assessment after natural infection or vaccination [22] .

The WHO IS is a pool of eleven human plasma from convalescent patients and was established in December 2020 by the WHO Expert Committee on Biological Standardization with an arbitrary assigned unitage of 1000 International Unit per mL (IU/mL) for neutralization activity [23] [24] ; the role of the IS in binding antibody assays was discussed. Not enough clinical data were available to assign an International Unit for binding activity and it was suggested to report the results in Binding Antibody Units (BAU) to assist in comparing assays that detect the same class of immunoglobulins with the same specificity (e.g., anti-receptor-binding domain IgG, anti-N IgM, etc).

In the present work, we aimed to compare six commercial quantitative anti-S protein serological assays, all with the manufacturers' conversion factor for the WHO IS for anti-SARS-CoV-2 immunoglobulin, in a head-to-head comparison to verify in "real life" whether the WHO IS effectively harmonizes laboratory testing.

Eighty-eight consecutive serum samples (F:M ratio 1.4:1, mean age= 57.0±14.0 years) screened for anti-S protein antibodies at the Clinical Chemistry Laboratory of "Tor Vergata" University Hospital (Rome, Italy), were tested using six different commercial kits widely used worldwide for anti-SARS-CoV-2 IgG or total antibodies (tAb) detection. Our study cohort reasonably consisted of a mixed population (vaccinated subjects, recovered patients from COVID-19, and healthy individuals) and no longitudinal samples were included. All assays were performed according to the manufacturers' cutoff and instructions.

The characteristics of the different serological tests evaluated are listed in Table 1 , including detailed information about targeted antigens. The linear dynamic ranges have been extended conforming to the manufacturers'instructions.

The study was performed according to local ethical approval protocol no. R.S.44.20. Informed consent was obtained from all subjects enrolled in the study. The study was in accordance with the Helsinki Declaration, as revised in 2013.

The Maglumi SARS-CoV-2 S-RBD IgG (Snibe S-RBD IgG) is an indirect chemiluminescence immunoassay (CLIA) for the in vitro quantitative determination of IgG antibodies to SARS-CoV-2 S-RBD protein, performed by fully automated Maglumi 800 analytical system (Snibe Diagnostic, Shenzhen, China). The samples, buffer solution and magnetic beads coated with S-RBD recombinant antigen are mixed thoroughly. After settling in a magnetic field, the supernatant is decanted and a wash cycle is performed. Then, anti-human IgG antibodies labelled with aminobutyl-ethyl-isoluminol (ABEI) are added and incubated to form immune-complexes. After a second precipitation in a magnetic field, the supernatant is decanted and another wash cycle is performed.

A starter reagent is added to initiate a chemiluminescent reaction, producing a light signal measured by a photomultiplier as Relative Light Units (RLUs) which is proportional to the concentration of anti-SARS-CoV-2 S-RBD IgG present in the sample.

The cut-off value in arbitrary units (AU)/mL, the conversion factor to obtain BAU/mL, the cut-off value in BAU/mL and the linearity range in AU/mL are respectively: 1, 4.33, 4.33 and 0.18-100, as declared by the manufacturer.

Samples with values over 100 AU/mL (433 BAU/mL) were diluted and measured 1:10 or 1:20 (if necessary), allowing extension of the dynamic range of analysis to 2000 AU/mL (8660 BAU/mL).

The manufacturer states intra-and inter-assay precision between 1% and 4%, with a clinical specificity of 99,6% (95% CI: 98,7%-100%) and a cumulative sensitivity of 100% (95% CI: 99,9%-100%), calculated at 15 days or more after the first positive PCR.

The Mindray SARS-CoV-2 S-RBD IgG (Mindray S-RBD IgG) is a two-step CLIA for quantitative The cut-off value in arbitrary units (AU/mL), the conversion factor to obtain BAU/mL, the cut-off value in BAU/mL and the linearity range in AU/mL are respectively: 0.8, 1.0288, 0.823 and 0.40-250, as declared by the manufacturer.

Samples with values over 250 AU/mL (257.2 BAU/mL) were diluted and measured 1:50, allowing the extension of the dynamic range of analysis to 12550 AU/mL (12911 BAU/mL).

The manufacturer states intra-and inter-assay precision between 1.4% and 2.4%, with a clinical specificity of 99.9% (95% CI: 99.9%-100%) and a cumulative sensitivity of 98,8% (95% CI:

98.1%-99.3%), calculated after 14 days or more from symptoms onset.

The EliA SARS-CoV-2-Sp1 IgG (Thermo Fisher S1 IgG) is a fluoroenzyme-immunoassay (FEIA)

for the in vitro quantitative measurement of IgG antibodies in human serum and plasma directed to the SARS-CoV-2 spike 1 protein, performed on the Phadia 250 instrument (Thermo Fisher, Uppsala, Sweden).

The EliA SARS-CoV-2-Sp1 IgG wells are coated with recombinant SARS-CoV-2 S1 protein.

After washing non-bound antibodies, enzyme-labelled anti-human IgG are added to form antibodyconjugate complexes incubated with a development solution. After stopping the reaction, the fluorescence in the reaction mixture is measured.

The cut-off value in AU/mL, the conversion factor to obtain BAU/mL, the cut-off value in BAU/mL and the linearity range in AU/mL are respectively: 7.0, 4.0, 28 and 0.7-204, as declared by the manufacturer.

Samples with values above 204 AU/mL (816 BAU/mL) were diluted and measured 1:2 allowing the extension of the dynamic range up to 408 U/mL (1632 BAU/mL).

According to the manufacturer, intra-and inter-assay precision data ranges between 2.1% and 3.1%;

clinical specificity is 99.7% (95% CI: 98.4%-100%) and clinical sensitivity is 100% (95% CI:

99.5%-100%), calculated at 8 days or more, after the first positive PCR.

The The cut-off value in AU/mL, the conversion factor to obtain BAU/mL, the cut-off value in BAU/mL and the linearity range in AU/mL are respectively 13.0, 2.6, 33.8, 1.84-800, as declared by the manufacturer.

Samples with values above 800 AU/mL (2080 BAU/mL) were not allowed to be diluted at the date of the experiment (March 20,2021).

According to the manufacturer, intra-and inter-assay precision ranges between 3.6% and 9%.;

clinical specificity is 99.5% (95% CI: 99.0%-99.7%) and clinical sensitivity is 98.7% (95% CI:

94.5%-99.6%), calculated at 15 days or more, after the first positive PCR.

The anti-SARS-CoV-2 QuantiVac enzyme immunosorbent assay (ELISA) IgG (Euroimmun S1

IgG) is a commercially available quantitative immunoassay test system (Euroimmun, Lübeck, Germany). The QuantiVac ELISA detects IgG antibodies from serum, plasma, or dried blood spot (DBS) using the S1 domain of the recombinant SARS-CoV-2 spike protein expressed in the HEK 293 human cell line. In the first step, diluted samples are incubated in the wells coated with S1. The specific IgG antibodies bind to the S1 antigens and are detected after a second incubation step, performed using an enzyme-labelled anti-human IgG to catalyze a color reaction. The absorbancies

are measured with a standard microplate reader at a wavelength of 450 nm. A standard curve obtained by the calibrators optical densities is used to assess the concentration of antibodies in the samples.

The cut-off value in AU/mL, the conversion factor to obtain BAU/mL, the cut-off value in BAU/mL and the linearity range in AU/mL are respectively 8.0, 3.2, 25.6, 6-120, as declared by the manufacturer.

Samples with values above 120 AU/mL (384 BAU/mL) were diluted and measured 1:10 allowing the extension of the dynamic range up to 1200 AU/mL (3840 BAU/mL).

According to the manufacturer, intra-and inter-assay precision ranges between 6.3% and 9.1%; clinical specificity is 99.8% and clinical sensitivity is 90.3%, calculated at 21 days or more, after symptoms onset.

Analyses were performed using MedCalc 19. 

antibodies assays and values were compared, expressing the results in both kit's arbitrary unit (AU)/mL and BAU/mL (Supplemental Data Table S1 ). Table 1 illustrates the main features of each serological assay related to its target antigen, method, immunoglobulin class detected, cut-off (AU/mL), conversion factor, cut-off (BAU/mL) and dynamic range, according to the manufacturer's instructions. In Figure 1 all data, detected by the serological assays in BAU/mL, were gradually sorted from the lowest to the highest value and have been shown as scatter-plot.

Since Roche method has the highest linearity range (0-12911 BAU/mL), and is able to detect total antibodies to the SARS-CoV-2 S-RBD protein, all values for each data set have been analyzed following the Roche results trend. Data have the best linear fit for values under 1000 BAU/mL and they scattered over 1000 BAU/mL showing some discrepancies. In fact, the enlargement of Fig. 1 shows a good alignment up to 1000 BAU/mL (ICC = 0.71) which then begins to disperse, becoming extremely evident over 1000 BAU/mL (ICC = 0.48). Moreover, it should also be considered that some serological assays reached a plateau, represented by the upper value of their linearity range allowed.

In Table 2 a correlation matrix shows the correlation coefficients for the serological assays. In each cell the correlation factors (R), the significance levels (p) and the number of the samples (n) have been reported. In particular, the lowest correlation coefficient value was obtained from Roche S-RBD tAb -DiaSorin Trimeric S IgG comparison (R=0.513); the highest correlation coefficient value was obtained from Euroimmun S1 IgG-Thermo Fisher S1 IgG comparison (R=0.942). Interestingly, on the one hand the highest correlation coefficients were obtained comparing methods with the same target antigen and isotype (S1 IgG for Euroimmun-Thermo Fisher, R=0.942; S-RBD IgG for Snibe-Mindray, R=0.940); on the other hand a different target antigen and/or antibody isotype gave a moderate correlation coefficient (S-RBD tAb for Roche and Trimeric S IgG for DiaSorin, R=0.513). All data were statistically significant (p<0.001).

Furthermore, the correlation data have been represented in Fig. 2 . The linear fit of the correlation rate between two assays, is shown by a straight line and the analysis was performed only considering the detectable values, excluding all samples outside the linearity range of each method.

Lastly, Table 3 highlights the values (k) obtained through Cohen's Kappa analysis. All the k values agreed significantly (p<0.001) ranging from substantial (0.711) to perfect (1.00), and with the only exception of the pairwise agreement between Snibe S RBD IgG and Thermo Fisher S1 IgG which revealed a moderate agreement (k=0.580). When between-assays variability was assessed, the difference between the median value of the BAU/mL CV (0.38; IQR: 0.23) and the median value of the (AU)/mL CV (0.68; IQR: 0.26) was statistically significant (p<0.001).

if immunological memory can be maintained over time in the absence of measurable levels of serum antibodies, serological assays could be used as surrogates to determine the protection of infection and vaccine-induced immunity to SARS-CoV-2. In a clinical laboratory they also increasingly represent an important element for long-term strategies, as for the recent hypothesis of supplemental vaccine doses [25] .

Since S protein constitutes the primary target of all current vaccines, serological assays are increasingly geared towards mainly using this protein or parts of it; also the S protein is likely to be the target of neutralizing antibodies, particularly the RBD. Nevertheless, it is still unclear exactly which antigen(s) would be preferable for the antibody assays in the disease and vaccine monitoring.

Analogously to what happens for vaccines, a specific assay might in fact not work well, and starts producing more false negatives, because it no longer binds to the target peptides as they have mutated in the emerging virus variants. In addition, it is evident from the literature that variability among different assays exist, due to the different method architecture, antigen to target, test antibody quality, and isotype of antibodies to be detected [26] [27] . Several studies have been recently published on this topic comparing SARS-CoV-2 serological assays focusing mainly on the the sensitivities of the different kits at different time intervals from symptoms' onset and therefore evaluating the best sensitive method for COVID-19 instead of antibodies levels [28] [29] [30] [31] .

In a 2020 collaborative study, the use of the WHO International Standard showed a reduction in the results reported from different laboratories worldwide, of 2000-fold for ELISA methods and more than 50 times for neutralization assays [32].

The authors have clearly underlined in the document that despite the strong harmonisation observed in the collaborative study results, even between binding assays targeting different viral antigens, that may not reflect what happens in a diagnostic laboratory. Indeed, the study samples were mainly pools of samples collected before May 2020 at similar time points post symptomps. More data are required and until then, as included in the WHO instructions for use, the results expressed in BAU/mL have to be antigen specific and serological assay targeting different antigens (e.g. RBD versus S) are not necessarily harmonised, even after recalculation in BAU/mL.

testing considers all phases, from pre-analytical to post-analytical, while the standardization is a specific process, referred as assay calibrator and calibration development [33] . To this matter, each manufacturer should indicate the complete methodological traceability chain and lot-to-lot variation in terms of measurement uncertainty, according to the EN ISO 17511:2020 (en) [34] .

However, when the first WHO 20/136 IS became available, none of the manufacturers adopted the new standardization on the existing assays, but through experimental tests provided a conversion factor (AU/mL to BAU/mL) to create a harmonization.

In the current study we performed a head-to-head comparison of six commercially available anti-SARS-CoV-2 serological tests, confirming the effect of the WHO IS in reducing the results variability (Supplemental Data Table S1 ). The main limitation of the the study is related to the sample size reduced in some statistical analysis because only detectable values could be taken into account. Moreover, the highest dispersions observed by the comparison of curves for values >1000 AU/mL may be explained by the different dynamic ranges among methods, whereby some of them had reached a plateau. In detail, when comparing assays with various measuring ranges, it is important to consider the specific concentrations assigned to the calibrators.

Furthermore, the intrinsic variability of each specimen should be considered. In fact, due to differing binding characteristics of the antibodies in patients' sera, not all the samples can be diluted linearly within the measuring range.

We analyzed serological assays in pairs and we found in some cases a very good correlation (i.e.

Euroimmun S1 IgG vs Thermo Fisher IgG: ρ= 0.942) and in other cases significant differences (TriS DiaSorin IgG vs Roche S-RBD tAb: ρ= 0.513). Interestingly, since the lowest agreement values were found when kits using different target antigens or different antibody isotypes were compared, these discrepancies might be due mainly to the different target antigens used for the assays (S1, S-RBD, trimeric S) and/or to the immunoglobin classes detected (IgG or total IgG/IgA/IgM), as well as the detection methods based on several technologies (fluorescence, chemiluminescence, electrochemiluminescence). A similar finding was observed by Van Elsande et al., where the agreement between the two IgG anti-S assays (0.915) was higher than between an anti-N assay and anti-S assay (0.839-0.901) [35] . This further makes clear why it is advisable that the laboratory report should include the information on the target antigen and immunoglobulin isotype of the assay kit.

The average agreement among methods for positive/negative results assessed by the Cohen's Kappa was almost perfect (κ=0.820), better than the average of correlation coefficients (ρ=0.786).

Hence, this could signify that today the immunological response to the virus should be interpreted mainly in terms of positive/negative. Then, only after the harmonization and standardization processes have moved on, would it make sense to compare antibody levels in patients or establish a precise threshold for immunity, even when tested by different manufacturer's assays.

At present anti-SARS CoV-2 serological assays' results are not interchangeable and individual immune monitoring should be performed using the same method.

The expression in BAU/mL may allow: 1) a first important step in the harmonization process, reducing differences among reported values compared to the expression in AU/mL; 2) a common language to report antibodies results.

The WHO, the CDC and the European Commission's Joint Research Centre (JRC) have worked and are still working, in synergy with manufactures, to further improve harmonization and to reach the standardization of SARS-CoV-2 quantitative antibodies and of the neutralization assays.

The authors have declared no conflict of interest. 

EliA SARS-CoV-2-Sp1 IgG S1 FEIA IgG 7 (7-10 borderline) 4

QuantiVac ELISA IgG S1 ELISA IgG 8 (8-11 borderline) 3.2 Table 1 Characteristics of anti-SARS-CoV-2 serological assays: target antigen, method, immunoglobulin class detected, cut-off value in arbitrary units (AU/mL), BAU conversion factor, cut-off value in BAU/mL and dynamic range, as declared by the manifacturer.

CLIA, chemiluminescence immunoassay; ECLIA, electrochemiluminescence immunoassay;

ELISA, enzyme-linked immunosorbent assay; FEIA, fluorescence enzyme immunoassay. Table 3 . Pairwise Cohen's Kappa agreement among the six serological assays (95% confidence intervals and level of significance). All data were statistically significant with p<0.001.

Serology assays to manage covid-19

Serological Assays for SARS-CoV-2 Infectious Disease: Benefits, Limitations and Perspectives

SARS-CoV-2 infection serology: a useful tool to overcome lockdown?

COVID-19) emergency use authorizations for medical devices: in vitro diagnostics EUAs

SARS-COV-2 diagnostic pipeline

COVID-19 Target product profiles for priority diagnostics to support response to the COVID-19 pandemic v.0.1; 2020

SARS-CoV-2 seroprevalence in COVID-19 hotspots

ENE-COVID Study Group. Prevalence of SARS-CoV-2 in Spain (ENE-COVID): a nationwide, population-based seroepidemiological study

Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease 2019

Profiling Early Humoral Response to Diagnose Novel Coronavirus Disease (COVID-19)

Assay Techniques and Test Development for COVID-19 Diagnosis

Detection of SARS-CoV-2-Specific Humoral and Cellular Immunity in COVID-19 Convalescent Individuals

Treatment of 5 Critically Ill Patients With COVID-19 With Convalescent Plasma

Antibodies for Prevention and Treatment of COVID-19

Human neutralizing antibodies elicited by SARS-CoV-2 infection

Immunological considerations for COVID-19 vaccine strategies

Prospect of SARS-CoV-2 spike protein: Potential role in vaccine and therapeutic development

Humoral Immune Response to SARS-CoV-2 in Iceland

SARS-CoV-2 Serologic Assay Needs for the Next Phase of the US COVID-19 Pandemic Response

COVID-19) Emergency Use Authorization for Medical Devices

SARS-CoV-2 Serologic Assay Needs for the Next Phase of the US COVID-19 Pandemic Response

First WHO International Standard for anti-SARS-CoV-2 immunoglobulin

Will SARS-CoV-2 Infection Elicit Long-Lasting Protective or Sterilising Immunity? Implications for Vaccine Strategies

Antibody response to SARS-CoV-2 infection in humans: A systematic review

Diagnostic accuracy of serological tests for covid-19: systematic review and meta-analysis

Comparison of SARS-CoV-2 serological tests with different antigen targets

Comparison of different serological assays for SARS-CoV-2 in real life

Comparison of Serological Assays for the Detection of SARS-CoV-2 Antibodies

Evaluation of SARS-CoV-2 serology assays reveals a range of test performance

Harmonization of laboratory testing -Current achievements and future strategies

Antibody response against SARS-CoV-2 spike protein and nucleoprotein evaluated by four automated immunoassays and three ELISAs

Conceptualization, Investigation, Formal analysis, Methodology, Visualization, Writing -original draft, Writing -review & editing

Formal analysis, Methodology, Visualization, Writing -original draft, Writing -review & editing

Massimo Pieri: Conceptualization, Investigation, Formal analysis, Methodology, Visualization, Writing -original draft, Writing -review & editing

Formal analysis, Methodology, Visualization. Flaminia Tomassetti: Formal analysis, Methodology, Visualization

Graziella Calugi: Formal analysis, Methodology, Visualization

Formal analysis, Statistical analysis, Methodology, Visualization. Maurizio Benucci: Formal analysis, Methodology, Visualization

Formal analysis, Methodology, Visualization

Conceptualization, Visualization

Mariangela Manfredi: Conceptualization, Visualization, Supervision

Conceptualization, Visualization

The authors wish to thank Giada Mattiuzzo and Mark Page (National Institute for Biological Standards and Control, UK) for their critical reading of the manuscript and valuable comments.