key: cord-0919216-t29q6bgh
authors: Padoan, Andrea; Bonfante, Francesco; Pagliari, Matteo; Bortolami, Alessio; Negrini, Davide; Zuin, Silvia; Bozzato, Dania; Cosma, Chiara; Sciacovelli, Laura; Plebani, Mario
title: Analytical and clinical performances of five immunoassays for the detection of SARS-CoV-2 antibodies in comparison with neutralization activity
date: 2020-11-04
journal: EBioMedicine
DOI: 10.1016/j.ebiom.2020.103101
sha: 8d48d3ba0ff8aeeb7fad8c19baade1c888c1e984
doc_id: 919216
cord_uid: t29q6bgh

BACKGROUND: Reliable high-throughput serological assays for SARS-CoV-2 antibodies are urgently needed for the effective containment of the COVID-19 pandemic, as it is of crucial importance to understand the strength and duration of immunity after infection, and to make informed decisions concerning the activation or discontinuation of physical distancing restrictions. METHODS: In 184 serum samples from 130 COVID-19 patients and 54 SARS-CoV-2 negative subjects, the analytical and clinical performances of four commercially available chemiluminescent assays (Abbott SARS-Cov-2 IgG, Roche Elecsys anti-SARS-CoV-2, Ortho SARS-CoV-2 total and IgG) and one enzyme-linked immunosorbent assay (Diesse ENZY-WELL SARS-CoV-2 IgG) were evaluated and compared with the neutralization activity achieved using the plaque reduction neutralization test (PRNT). FINDINGS: Precision results ranged from 0.9% to 11.8% for all assays. Elecsys anti-SARS-CoV-2 demonstrated linearity of results at concentrations within the cut-off value. Overall, sensitivity ranged from 78.5 to 87.7%, and specificity, from 97.6 to 100%. On limiting the analysis to samples collected 12 days after onset of symptoms, the sensitivity of all assays increased, the highest value (95.2%) being obtained with VITRO Anti-SARS-CoV-2 Total and Architect SARS-CoV-2 IgG. The strongest PRNT(50) correlation with antibody levels was obtained with ENZY-Well SARS-CoV-2 IgG (R(2)adj = 0.569). INTERPRETATION: The results confirmed that all immunoassays had an excellent specificity, whereas sensitivity varied across immunoassays, depending strongly on the time interval between symptoms onset and sample collection. Further studies should be conducted to achieve a stronger correlation between antibody measurement and PRNT(50) in order to obtain useful information for providing a better management of COVID-19 patients, effective passive antibody therapy, and developing a vaccine against the SARS-CoV-2 virus. FUNDING: None.

The continuing spread of coronavirus disease 2019 (COVID- 19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has prompted concern worldwide, leading the World Health Organization (WHO) to declare COVID-19 a pandemic on 11 March 2020 [1] . The accurate and timely diagnosis of the disease is crucial to the effective management of patients, control of the pandemic and the establishment of appropriate infection control measures.

Although real-time reverse transcription polymerase chain reaction (rRT-PCR) allows the diagnosis of the disease in most patients, including asymptomatic carriers, it has some analytical and clinical limitations. Analytical pitfalls in both the pre-and analytical steps have been described [2] and negative molecular test results have been reported in the later stages of infection, thus being misleading from a clinical viewpoint. Therefore, rRT-PCR precludes the identification of individuals who have been infected, but have had only minor, or no, symptoms and therefore have not sought medical attention. A wide range of immunoassays to detect SARS-CoV-2 antibodies (Ab) have been developed to complement rRT-PCR, with different antigen targets and formats [3À5] . Although not well suited for allowing an early diagnosis, serological assays for SARS-CoV-2 may play an important role in diagnosing COVID-19 disease in individuals who present late, in understanding the virus epidemiology in the general population, and in identifying the disease prevalence in categories at higher risk of infection (e.g. healthcare workers) [6] . In addition, they should be used to ascertain the efficacy of containment measures both locally and globally, to screen convalescent sera for therapeutic and prophylactic purposes, and to improve knowledge of the immune response to the novel virus as the degree and duration of the response of specific antibodies is as yet poorly understood [7, 8] . Like infections from other pathogens, SARS-CoV-2 infection elicits development of IgM and IgG specific Ab which are the most available antibodies for assessing response, while little is known about IgA response in the blood.

The aim of this paper is to evaluate the performance characteristics and diagnostic specificity, sensitivity of four chemiluminescent assays (CLIA) and one enzyme-linked immunosorbent assay (ELISA) for SARS-CoV-2 antibodies and in comparison with neutralizing activity.

A total of 184 leftover serum samples from 130 COVID-19 patients (8 asymptomatic/paucisymptomatic who recovered at home with supportive care and isolation, and 122 hospitalized classified with moderate or severe disease, following WHO interim guidance [9] ) and 54 SARS-CoV-2 negative subjects (33 healthcare workers, 21 autoimmune patients, 8 pregnant women) were included in the study (Table 1 ). All subjects underwent at least one nasopharyngeal swab test, analyzed by rRT-PCR. Healthcare workers were considered negative (NHW) on the basis of at least three negative sequential molecular test results obtained between February 26th and May 29th, 2020. Raw data of the study is available at 10.6084/m9.figshare.12928832.

In this study, an evaluation was made of four commercially available chemiluminescent immunoassays (CLIA) [ 

Evidence before this study

We searched Pubmed (NCBI) on July 13, 2020, for articles published with the keywords ("SARS-CoV-2" OR "COVID-19") AND ("antibody" OR "antibodies") AND ("neutralization" OR "neutralisation" OR "neutralizing" OR "PRNT") AND ("performance" OR "performances" OR "evaluation" OR "clinical" OR "comparison") in humans. We did not restrict our search to a publication language. This search retrieved a total of 61 papers, some of them dealing with the comparison of the clinical performances of anti-SARS-CoV-2 Ab immunoassays to neutralization titers. One report compares IgG or total antibodies (AbT) measurement of three enzymelinked immunosorbent assay (ELISA), two CLIA chemiluminescent assays (CLIA) and one enzyme-linked immunosorbent assay (ELISA), in a total of 100 SARS-CoV-2 convalescent plasma donors and found a good correlation (rho > 0.700) between ELISA (Euroimmun IgG and Wantai Total antibodies) and neutralization titer. Another study evaluated the performances of six commercial immunoassays for the detection of IgG, IgA and IgM antibodies, including four CLIA automated assays [Abbott SARS-COV-2 IgG, Diasorin Liaison IgG and Euroimmun SARS-COV-2 IgG and IgA], and two RLF [Acro Biotech 2019-nCoV IgG/IgM and Xiamen Biotime Biotechnology SARS-COV-2 IgG/IgM] with a microneutralization test (MNT). Evaluating 70 included sera from PCR confirmed COVID-19 patients, a panel of 81 sera from negative subjects and patients with autoimmune disease and with respiratory virus, a total of forty-one out of 62 COVID-19 patients showed neutralizing antibodies. Another study evaluated two ELISA assays (Euroimmun SARS-CoV-2 IgG and Vircell COVID-19 ELISA IgG), one LFA (FaStep COVID-19 IgG/IgM Rapid Test Device) and two in-house developed assays. For all examined assays, the sensitivity ranged from 58.8 to 76.5% for the early phase of infection (days 5-9) and from 93.8 to 100% for the later period (days [10] [11] [12] [13] [14] [15] [16] [17] [18] . Four automated immunoassays (Abbott Architect, Roche Cobas, LIAISON, VIRCLIA automation system) in comparison to two ELISA assays (Euroimmun SARS-CoV-2 IgG and Virotech SARS-CoV-2 IgG ELISA) an in-house developed plaque reduction neutralization test (PRNT) were tested with serum/plasma samples of followed up individuals with PCR-diagnosed COVID-19. A highest sensitivity of 93.3% was achieved, whilst the specificity of the examined assays was 97%.

In this study, a series of four CLIA and one ELISA immunoassays were examined for clinical performances and for the correlation of anti-SARS-CoV-2 Ab serum levels with the gold standard method for determining PRNT. According to suggestions reported in some recent meta-analyses, this study evaluated immunoassays performances in different time frames, including the early Ab response. Furthermore, different groups of subjects were included to derive robust estimations (negative subjects included autoimmune patients and pregnant women, while SARS-CoV-2 positive included patients with asymptomatic/paucisymptomatic, moderate and severe diseases). Besides to evaluate the clinical performances of the examined immunoassays, this study was designed to collect evidence on the correlation between neutralization activity and anti-SARS-CoV-2 Ab levels.

Comparative data for immunoassays is needed as a basis for the production of convalescent plasma, for potential interpretations COVID-19 immunity and for planning national and international strategies to reduce viral spread. To these purposes, data collected from multiple studies are required to obtain an unbiased estimate of the current evidences, whilst careful study design is requested for pooling data. Taking all together, findings showed that commercial immunoassays performances are comparable and high positive or negative predictive values are achievable, especially from samples collected 12 days post symptoms onset. Furthermore, the poor non-linear response between neutralization activity and anti-SARS-CoV-2 Ab levels suggests that immunoassays can be mainly developed for detecting positive/negative subjects and for improving rRT-PCR diagnosis of COVID-19. These results can therefore be helpful for in vitro diagnostic device manufacturers, to develop more specific immunoassays.

Sallugia-VC, Italy), ENZY-Well SARS-CoV-2 IgA (ref 91402, IO09/442 v04-24-2020) and IgM (ref 91401, insert IO09/441 v04-24-2020) were evaluated for the correlation with the neutralization results.

Precision estimation was performed on CLIA assays using two human serum sample pools with different values, by means of quintuplicate measurements of same pool aliquots, performed for a total of four consecutive days. Nested analysis of variance was used to estimate precision, following the CLSI EP15-A3 protocol [10] . The results for precision were compared to those claimed by the manufacturer when available, using the procedure recommended by EP15-A3. Repeatability and within-laboratory precision were in accordance with the repeatability and intermediate precision conditions specified in the international vocabulary of metrology (VIM, JCGM 100:2012) for precision estimation within a four-day period.

Linearity was assessed using serial dilution of two samples pools, prepared with two different levels of SARS-CoV-2 antibodies (high and low level pools), as specified in the CLSI EP06-A, guideline (paragraph 4.3.1) [11] . In brief, the following high-level serum pools were prepared: 5.2 signal to cut-off (S/CO) ratio for VITROS Anti-SARS-CoV-2 IgG, 53 S/CO ratio for VITROS Anti-SARS-CoV-2 total, 5.4 S/CO ratio for Elecsys Anti-SARS-CoV-2, and 3.76 S/CO ratio for Architect SARS-CoV-2 IgG. The pools were serially diluted with the corresponding low-level serum pools (0.174 S/CO ratio for Elecsys Anti-SARS-CoV-2, 0.01 S/CO for VITROS Anti-SARS-CoV-2 IgG, 0.2 S/CO for VITROS Anti-SARS-CoV-2 total, 0.04 S/CO ratio for Architect SARS-CoV-2 IgG). All measurements were performed in triplicate.

For a subgroup of 68 samples from SARS-CoV-2 positive individuals, PRNT test was performed. These samples were randomly selected from the 130 SARS-CoV-2 positive individuals included in the study and independently from disease severity (Fisher's exact test, p = 0.103). Each sample was obtained from a different individual. On the same samples, the ENZY-WELL SARS-CoV-2 IgA and IgM (Diesse Diagnostica Senese, Siena, Italy) assays were further performed. The high-throughput method for PRNT was developed for the fast and accurate quantification of neutralizing antibodies in plasma samples collected from patients exposed to SARS-CoV-2. Samples were heatinactivated by incubation at 56°C for 30 min and 2-fold dilutions were prepared in Dulbecco modified Eagle medium (DMEM). The dilutions, mixed to a 1:1 ratio with a virus solution containing 20À25 focus-forming units (FFUs) of SARS-CoV-2 (self-obtained from patient isolate), were incubated for 1 h at 37°C. Fifty microliters of the virusÀserum mixtures were added to confluent monolayers of Vero E6 cells, in 96-wells plates and incubated for 1 h at 37°C, in a 5% CO 2 incubator. The inoculum was removed and 100 ml of overlay solution of Minimum essential medium (MEM), 2% fetal bovine serum (FBS), penicillin (100 U/ml), streptomycin (100 U/ml) and 0.8% carboxy methyl cellulose was added to each well. After 26 h' incubation, cells were fixed with a 4% paraformaldehyde (PFA) solution. Visualization of plaques was obtained with an immunocytochemical staining method using an anti-dsRNA monoclonal antibody (J2, 1:10,000; Scicons) for 1 hour, followed by 1 h incubation with peroxidase-labeled Table 2 The five SARS-CoV-2 antibody assays investigated: characteristics specified by the manufacturers. goat anti-mouse antibodies (1:1000; DAKO) and a 7 min incubation with the True Blue TM (KPL) peroxidase substrate. FFUs were counted after acquisition of pictures at a high resolution of 4800 £ 9400 dpi, on a flatbed scanner. Biosafety Level 3 laboratory setting was used for PRNT tests. The serum neutralization titer was defined as the reciprocal of the highest dilution resulting in a reduction of the control plaque count >50% (PRNT 50 ). To define a seropositive threshold and the specificity of the PRNT test, we included in our analyses 43 negative control sera collected in 2018, before the COVID-19 pandemic. To infer the inter-operator reproducibility of the PRNT assay, a subset of 29 sera from the 68 obtained from SARS-CoV-2 positive subjects was tested three times, by three different couples of operators, 1 to 2 weeks apart. An intraclass correlation coefficient was inferred as previously described elsewhere [12] . Liaison SARS-CoV-2 S1/S2 IgG (Diasorin, Sallugia, VC, Italy) [5] assay was further performed for 52 out of these 68 samples tested neutralization activity.

For evaluation of precision, an in-house developed R (R Foundation for Statistical Computing, Vienna, Austria) script for implementing the CLSI EP15-A3 protocol was used for ANOVA and for calculating the upper verification limit [10] . The GraphPad Prism version 8.4.1 for Windows (GraphPad Software, LLC) was employed to evaluate plaque reduction neutralization test results, using nonparametric tests (Kruskall-Wallis test and Spearman's correlation). MedCalc Statistical Software version 19.2.1 (MedCalc Software Ltd, Ostend, Belgium) was used for power analyses. Stata v13.1 (Statacorp, Lakeway Drive, TX, USA) was used to evaluate the assays' clinical performances, and for multivariate regressions. Bonferroni's adjusted p-value (B-adj) was calculated for multiple comparisons. For ROC analyses, a/the non-parametric empirical method was used to estimate the area under the ROC curve (AUC), while the 'diagt' module was used to estimate sensitivity, specificity, and positive and negative predictive values. Cohen's kappa was used to evaluate between methods agreements. Considering a type I error a = 0.05, a power of 0.8 and with 130 positive and 50 negative subjects, AUC above 0.95 can be considered significant with respect to values below 0.89 (null hypothesis).

The study protocol (number 23307) was approved by the Ethics Committee of the University-Hospital of Padova. All the patients were informed of the study and voluntarily agreed to participate. All the patients who agreed to participate provided written consent.

This study does not receive any specific grant from funding agency in the public, commercial, or not-for-profit sectors. Table 1 reports the demographic characteristics of the subjects included in the study. The overall mean age of subjects was 56.9 years, with a standard deviation ( §SD) of 19.1 (range 22.7 -92.2 years). Negative healthy workers (NHW) [Bonferroni's adjusted pvalue (B-adj) p < 0.0001], autoimmune/pregnant subjects (AI/Pr) (Badj p < 0.0001) and asymptomatic/paucisymptomatic (Asympt/ Pauci) subjects (B-adj p < 0.0001) were significantly younger than SARS-CoV-2 patients. The percentage of females differed significantly from that of males (p < 0.001), particularly in the AI/Pr group. For SARS-CoV-2 patients, the mean time interval from the onset of symptoms and serological determinations was 24.6 days (SD §18.6; range 4 -89 days) ( Table 3) .

Results for precision of all the CLIA assays are reported in Table 4 . The ANOVA approach allowed us to estimate repeatability and intermediate precision separately. Only the Architect SARS-CoV-2 IgG insert reported data on precision, claimed at levels of 0.04 and 3.53 S/CO ratio. For this immunoassay, intermediate precision performances statistically deviated from the manufacturer's claims at both levels. All the immunoassays had acceptable analytical imprecision (CV%).

Linearity results for all the CLIA studies are summarized in Fig. 1 . All tested mixes of sample pools covered a wide range of values and included the manufacturers' cut-offs. All immunoassays, except for Elecsys Anti-SARS-CoV-2, deviated from linearity, the coefficients of the second-order polynomial fit attaining high statistical significance.

Sensitivities, specificities, and positive and negative likelihood ratios were estimated using the manufacturers' cut-offs, while receiver operating characteristic (ROC) curves were used to evaluate Table 3 Disease severity, time from symptoms onset, percentage of positive samples to serological determination of SARS-CoV-2 antibodies and PRNT 50 titers, subdivided by the studied groups. overall performance(s). Elecsys Anti-SARS-CoV-2 immunoassay results were available for 172/184 (93.4%) serum samples. Table 5 summarizes estimated clinical performances for all CLIA and the ELISA immunoassays considering the total time frame of 93 days and limiting the analyses to sera collected 12 days after the onset of symptoms. One hundred fifty-eight samples were included and evaluated in this restricted subgroup, while only 146 results were available for Elecsys Anti-SARS-CoV-2. Table 5 shows data on positive and negative likelihood ratios, allowing an easy estimation of positive (PPV) and negative (NPV) predictive values given disease prevalence.

Considering two different scenarios of disease prevalence settings: (a) 4%, as found in a Veneto Region (Italy) survey [13] ; (b) 10%, as described in a survey conducted in Geneva [14] , PPV and NPV were then estimated, using VITROS Anti-SARS-CoV- Table 4 Precision results for the studied immunoassays. Coefficient of variation (CV) are expressed in percentage (%) and were obtained by using pools of samples. obtained from the VITROS Anti-SARS-CoV-2 Total Ortho Clinical Diagnostic insert v4.0 pub. No. GEM1293_XUS_IT.°o btained from the Cobas Anti-SARS-CoV-2 Roche Diagnostic insert 09203095500 v1.0, 2020-05 in Italian. * indicates that imprecision value was higher than that declared by manufacturers, also after the calculation of UVL as suggested by EP15-A3 (5.4% for Repeatability and 5.9% for Intermediate precision at level 0.04 Index S/C and 1.1 for Repeatability and 1.2% for Intermediate precision at level 3.53 S/CO). considering the total time frame, these results changed to 100% (19.7À98.4%) and 99.4% (99À99.6%), and 100.0% (39.5À99.4%) and 98.3% (97.5À98.8%) with prevalence settings of 4% and 10%.

Since the results of serum samples and the corresponding immunoassays' cut-offs were used to derive either positive or negative test results, pairwise comparisons of all tests by Cohen's kappa and overall agreements (in percentages) were calculated considering overall data ( Table 6 ).

All the 43 negative control sera sampled in 2018 failed to neutralize virus (titers were < 1:10 for all samples) and, therefore, PRNT assay analytical sensitivity was estimated to be 100%. For this reason, we established that test sera recording titers equal to or above 1:10 should be considered as positive. Subsequently, among the 68 sera from COVID-19 patients, 62 resulted positive, recording values ranging from 1:10 to 1:5120. The signal-to-cut-off (S/CO) ratios of the examined assays, including Liaison SARS-CoV-2 S1/S2 IgG, ENZY-WELL SARS-CoV-2 IgA and IgM and the corresponding PRNT 50 titers for the 68 tested SARS-CoV-2 serum samples are shown in Fig. 2 . The associations between PRNT 50 titers and assays S/CO ratios were evaluated by multivariate analyses, using SARS-CoV-2 antibody levels, age, gender, time from symptom onset and disease severity as additional predictors. At multivariate analyses, the highest association was obtained with ENZY-WELL SARS-CoV-2 IgG (R 2 adj = 0.569) followed by VITROS Anti-SARS-CoV-2 IgG (R 2 adj = 0.544), Architect SARS-CoV-2 IgG (R 2 adj = 0.477), ENZY-WELL SARS-CoV-2 IgM (R 2 adj = 0.406), Liaison SARS-CoV-2 S1/S2 IgG assay (R 2 adj = 0.402), and ENZY-WELL SARS-CoV-2 IgA (R 2 adj = 0.241) ( Supplementary  Tables 1-8) . VITROS Anti-SARS-CoV-2 Total (R 2 adj = 0.117) and Elecsys Anti-SARS-CoV-2 (R 2 adj = 0.046) showed a very limited association with PRNT 50 titers. Inter-operator reproducibility of the PRNT assay, estimated using a subset of 29 sera from the 68 obtained from SARS-CoV-2 positive subjects, was supported by an intraclass correlation coefficient of 0.933.

The multivariate models, including Ab levels and Age, gender, time from symptoms onset and disease severity, demonstrate higher levels of SARS-CoV-2 Ab with increasing time from symptoms onset. No significant association was found between SARS-CoV-2 Ab levels and age, gender and disease onset (Supplementary Tables 9-13 and Supplementary Figures 1-2 

In the last few months, numerous SARS-CoV-2 serology assays have been developed. The complexity of COVID-19 has called for careful study design to obtain meaningful information, and some of the assays have not yet been extensively validated by independent laboratories. In a recent meta-analysis, it was pointed out that most studies on SARS-CoV-2 serology have assessed sensitivity without considering time from onset of symptoms and/or including COVID-19-positive cases that are rRT-PCR-negative [15] . Moreover, researchers are currently facing other knowledge gaps in SARS-CoV-2 serology. For example, the understanding of the neutralization activity of serum antibodies against viral particles is incomplete, calling for the development of strategies to improve our understanding of their relationship with SARS-CoV-2 Ab detected by conventional immunoassays [16] . Serological tests should be evaluated in parallel to neutralization assays, since not only these represent the gold standard in terms of assay specificity and may provide evidence of the mechanism of development of viral immunity, but also they are the only assays measuring the actual protective immunity of antibodies [17] . In this retrospective study, the analytical and clinical performances of four commercially available CLIA assays and one ELISA assay ( Table 2) have been evaluated and compared with neutralization activity using the plaque reduction neutralization test (PRNT). The neutralization activity was evaluated also with respect Liaison SARS-CoV-2 S1/S2 IgG and ENZY-WELL SARS-CoV-2 IgA and IgM. Before conducting the study, precision at two concentration levels and linearity were assessed for CLIA by using a standardized protocol according to the CLSI EP15-A3 and CLSI EP06-A (Table 4 and Fig. 1 ) [10, 11] The results obtained demonstrated that both repeatability and intermediate precisions were comparable with other immunoassays performances for the highest concentration levels [3, 5] , whilst for the lowest levels, less satisfactory results were obtained for VITROS Anti-SARS-CoV-2 Total and Architect SARS-CoV-2 IgG. However, precision results do not have clinical impact as a small variations of the S/CO will not modify the interpretation of the serology.

Linearity was assessed in a range of values covering manufacturers' cutoffs ( Fig. 1) . Linear results were obtained for Elecsys Anti-SARS-CoV-2, whereas the performance of other immunoassays was less effective, demonstrating that a double serum antibodies concentration will not correspond to a double in S/CO ratio. To assess the clinical performances of immunoassays, a total of 184 leftover samples obtained from Negative Healthy workers, Autoimmune patients/ pregnant women, asymptomatic/paucisymptomatic and Moderate/ Severe SARS-CoV-2 patients were evaluated ( Table 1 ). The 21 autoimmune patients and the eight pregnant women, who were SARS-CoV-2 negative, were included in order to evaluate possible analytical interferences. According to the suggestion on study design for SARS-CoV-2 serology, clinical performances were evaluated by considering the total time frame (overall data), and limiting the analyses to sera collected 12 days after the onset of symptoms as this period greatly impacts on immunoassay sensitivity [5] . Table 3 shows the distribution of sera among patient groups, the delay since symptom onset, the number and the percentages of samples tested positive for the SARS-CoV-2 antibodies assays and the PRNT 50 titer. Taking into consideration the time from symptoms onset and sera collection, significant differences were found between Asympt/Pauc and Mod (p < 0.01) and Asympt/ Pauc and Sev (p < 0.01). These differences might be at least in part due to the difficulty to obtain serum samples from Asympt patients because they were not hospitalized, but in quarantine at home. Table 3 shows that the percentages of samples with SARS-CoV-2 Ab positive results increased from Moderate to Severe disease. However, multivariate linear regression analyses confirmed that, among the studies variables (age, gender, disease severity and time from symptoms onset), only time from symptoms onset was associated, with an increasing trend, with SARS-CoV-2 Ab serum levels (Supplementary Tables 9-13 , Supplementary Figures 1-2) .

Results of ROC analyses showed overlapping performances for all immunoassays (Table 5) . VITROS Anti-SARS-CoV-2 Total provided the best results, with an AUC over 97% (lower confidence limit > 95%). Considering overall data, Elecsys Anti-SARS-CoV-2 and Architect SARS-CoV-2 IgG sensitivities obtained in our study are similar those reported by Kohmer et al. and Theel et al. [18, 19] . Sensitivities differed when only samples collected after 12 days post symptom onset were evaluated. This was expected since, after SARS-CoV-2 infection, antibody levels begin to rise as from the second week of onset of symptoms [20] . Evaluations after 12 days post symptom onset confirmed overall excellent results for all immunoassays, anti-SARS-CoV-2 Total and Architect SARS-CoV-2 IgG performances being the best. However, in contrast with statements in manufacturers' inserts, our data show that 100% true positive results cannot be obtained, even when the sample collection time is more than 14 days after onset of symptoms. These results are in agreement with findings reported by other Authors on comparing immunoassays results [17,21À23] . Asymptomatic/paucisymptomatic SARS-CoV-2 patients were correctly identified as positive by all immunoassays. These findings are promising, even if the number of evaluated patients is limited to draw robust conclusions. In fact, recent studies have shown that individuals with asymptomatic infections have lower antibody titers and a more rapid decline of those antibody titers, notably neutralizing antibodies [7, 24] . Differently, 95% CI of specificities reached 100% for all the assays, and these data are in line with manufacturers' inserts. Although excellent positive-(PLR) and negative-(NLR) likelihood ratios were obtained for all methods, Architect SARS-CoV-2 IgG PLR outperform all other immunoassays. PPV and NPV were further estimated considering two prevalence settings (4% and 10%). The highest achievable PPV and NPV values were 100% and 99.8% for PPV and NPV, respectively, obtained with Architect SARS-CoV-2 IgG and at a prevalence setting of 10%. These results are partially in agreement with that reported by Pfl€ uger, et al. in settings with a low prevalence [23] .

Overall, there was a good agreement between all assays, the better result being obtained for VITROS Anti-SARS-CoV-2 Total and ENZY-WELL SARS-CoV-2 IgG (Cohen's kappa = 0.943). Interestingly, the two methods differ both for the detection specificity (Total Ab vs IgG, respectively) and for viral protein targets (Spike protein S1 vs Native antigen).

To provide insights on neutralization activity compared with immunoassays results, PRNT assay was performed on 68 samples from SARS-CoV-2 positive subjects. With the exception of Elecsys Anti-SARS-CoV-2, other immunoassay results were correlated with PRNT 50 titer. These results agree with findings from Tang et al., who found a weak linear correlation between Elecsys Anti-SARS-CoV-2 Ab Tot levels and PRNT 50 titers [25] .

However, as shown in Fig. 2 , the dynamic range of measured antibodies, from the lowest to the highest PRNT 50 titer, is very limited, this being in line with the data reported by J€ a€ askel€ ainen and colleagues [17] . This might be due to several factors, such as: a) antibodies measurements may be specifically developed for detecting positive/negative subjects in order to improve rRT-PCR diagnosis of COVID-19, or b) a non-linear response between the real antibody concentration and instrumental signals. Notably, results of Liaison SARS-CoV-2 S1/S2 IgG assay were similar to those obtained with other immunoassays. At correlation analysis, no differences could be detected on comparing immunoassays developed against Spike or Nucleocapsid proteins. Differently, at multivariate analysis the strongest correlation with PRNT 50 titer was found for ENZY-WELL ELISA results, developed with a native antigen of SARS-CoV-2 (R 2 adj = 0.569, which correspond to a Pearson's rho of 0.753). Recently, Perera et al. found a slightly worse correlation (rho = 0.67) between plaque reduction neutralization results and an in-house developed IgG ELISA with recombinant RBD of the spike protein as coated antigen [26] . In addition, we found a moderate correlation between PRNT and IgM results, thus confirming the data reported by Perera and colleagues [26] . Another report, which compares IgG or total antibodies measurement of three ELISA, two CLIA and two lateral flow tests, in a total of 100 SARS-CoV-2 convalescent plasma donors, found a good correlation (rho > 0.700) between ELISA (Euroimmun IgG and Wantai Total antibodies) and neutralization titer [27] .

PRNT 50 titer were lower for Asympt/Pauc than Mod or Sev SARS-CoV-2 patients (Fig. 3, left panel) , although this difference was not statistically significant. Furthermore, no significant correlation was not found between the PRNT 50 titer and the time interval from symptom onset (Fig. 3, right panel) , whist a decreasing trend can be observed and could be verified in a more representative sample size.

The present paper has limitations: first, neutralizing antibodies were tested in a limited number of samples and sera were collected at various timepoints and, therefore, should be confirmed in further studies; second, COVID-19 positive patients were selected retrospectively on the basis of available leftover samples, and third cross-reactivity was not assessed with confirmed infection with other virus, namely other coronavirus; therefore NPV and PPV could be overestimated. Another limitation of this study is that no longitudinal sera were analyzed and, therefore, we cannot exclude that some patients might have seroconverted at later time points.

In conclusion, although the performances of SARS-CoV-2 antibody immunoassays are of analytical and clinical value, they could be enhanced by considering the test purposes, emphasizing sensitivity in the screening and specificity in the second-line testing. In addition, a further search should be made for a better dynamic range and a stronger correlation with respect to antibody neutralization activity, in order to, above all, obtain information needed for a better patient management, effective passive antibody therapy and vaccine development against SARS-CoV-2 virus. 

Timeline of WHO's response to COVID-19

Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19)

Analytical performances of a chemiluminescence immunoassay for SARS-CoV-2 IgM/IgG and antibody kinetics

IgA-Ab response to spike glycoprotein of SARS-CoV-2 in patients with COVID-19: A longitudinal study

Diagnostic performances and thresholds: the key to harmonization in serological SARS-CoV-2 assays?

Humoral Immune Response to SARS-CoV-2 in Iceland

Antibody responses to SARS-CoV-2 in patients with COVID-19

Serology assays to manage COVID-19

Clinical management of COVID-19, Interim guidance

User verification of precision and estimation of bias

Evaluation of the linearity of quantitative measurement procedures: a statistical approach; Approved guideline

Evaluation of accuracy and reliability of the plaque reduction neutralization test (micro-PRNT) in detection of yellow fever virus antibodies

Towards the rational utilization of SARS-CoV-2 serological tests in clinical practice

Towards the next phase: evaluation of serological assays for diagnostics and exposure assessment

Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane database Syst Rev 2020 (ahead of print)

Challenges in laboratory diagnosis of the novel coronavirus SARS-CoV-2

Performance of six SARS-CoV-2 immunoassays in comparison with microneutralisation

Brief clinical evaluation of six high-throughput SARS-CoV-2 IgG antibody assays

Performance characteristics of four highthroughput immunoassays for detection of IgG antibodies against SARS-CoV-2

Interpreting diagnostic tests for SARS-CoV-2

Clinical evaluation of serological IgG antibody response on the Abbott architect for established SARS-CoV-2 infection

Clinical performance of different SARS-CoV-2 IgG antibody tests shortened title: SARS-CoV-2 IgG antibody test performance

Clinical evaluation of five different automated SARS-CoV-2 serology assays in a cohort of hospitalized COVID-19 patients

Rapid decay of antiÀSARS-CoV-2 antibodies in persons with mild COVID-19

Association between SARS-CoV-2 neutralizing antibodies and commercial serological assays

Serological assays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)

Quantification of SARS-CoV-2 antibodies with eight commercially available immunoassays

We acknowledge Fondazione Cariparo (prot. n. 55813) to support this publication of the University of Padova, dealing with the COVID-19 pandemic. We thank Daniela Rinaldi (technical laboratory scientists) for their valuable technical support. We acknowledge Abbott Laboratories, Diesse Diagnostica Senese, Diasorin, Ortho Clinical Diagnostics, Roche Diagnostic for kindly supplying reagents without any influence in study design and data analysis.

Raw data of the study is available as a publicly created dataset for download at 10.6084/m9.figshare.12928832 (https://figshare.com/).

Supplementary material associated with this article can be found in the online version at doi:10.1016/j.ebiom.2020.103101.