key: cord-0844268-ftsus75q
authors: Moscato, Giovanna; Mazzetti, Paola; Lucenteforte, Ersilia; Rosellini, Alfredo; Cara, Alice; Quaranta, Paola; Mainardi, Valerio; Villa, Pietro; Focosi, Daniele; Lanza, Maria; Bianco, Irene; Mazzoni, Alessandro; Falcone, Marco; Menichetti, Francesco; Maggi, Fabrizio; Lai, Michele; Freer, Giulia; Pistello, Mauro
title: Assessment of automated high-throughput serological assays for prediction of high-titer SARS-CoV-2 neutralizing antibody
date: 2021-05-04
journal: Journal of clinical virology plus
DOI: 10.1016/j.jcvp.2021.100016
sha: 8a6f7e0f5f08f5a9950ae7190af2d50fc67de96b
doc_id: 844268
cord_uid: ftsus75q

COVID19 convalescent patient plasma units with high titer neutralizing antibody can be used to treat patients with severe disease. Therefore, in order to select suitable donors, neutralizing antibody titer against SARS CoV-2 needs to be determined. Because the neutralization assay is highly demanding from several points of view, a pre-selection of sera would be desirable to minimize the number of sera to be tested. In this study, a total of 140 serum samples that had been titrated for SARS-CoV-2 neutralizing antibody by microneutralization assay were also tested for the presence of anti-SARS-CoV2 antibody using 5 different tests: Architect® immunoassay (Abbott Diagnostics), detecting IgG against the nucleocapsid protein, LIAISON XL® (Diasorin) detecting IgG against a recombinant form of the S1/S2 subunits of the spike protein, VITROS® (Ortho Clinical Diagnostics), detecting IgG against a recombinant form of the spike protein, and ELISA (Euroimmun AG), detecting IgA or IgG against a recombinant form of the S1 subunit. To determine which immunoassay had the highest chance to detect sera with neutralizing antibodies above a certain threshold, we compared the results obtained from the five immunoassays with the titers obtained by microneutralization assay by linear regression analysis and by using receiver operating characteristic curve and Youden's index. Our results indicate that the most suitable method to predict sera with high Nab titer is Euroimmun® IgG, followed closely by Ortho VITROS® Anti-SARS-CoV-2 IgG.

would be desirable to minimize the number of sera to be tested. In this study, a total of 140 serum samples that had been titrated for SARS-CoV-2 neutralizing antibody by microneutralization assay were also tested for the presence of anti-SARS-CoV2 antibody using 5 different tests: Architect ® immunoassay (Abbott Diagnostics), detecting IgG against the nucleocapsid protein, LIAISON XL ® (Diasorin) detecting IgG against a recombinant form of the S1/S2 subunits of the spike protein, VITROS ® (Ortho Clinical Diagnostics), detecting IgG against a recombinant form of the spike protein, and ELISA (Euroimmun AG), detecting IgA or IgG against a recombinant form of the S1 subunit. To determine which immunoassay had the highest chance to detect sera with neutralizing antibodies above a certain threshold, we compared the results obtained from the five immunoassays with the titers obtained by microneutralization assay by linear regression analysis and by using receiver operating characteristic curve and Youden's index. Our results indicate that the most suitable method to predict sera with high Nab titer is Euroimmun ® IgG, followed closely by Ortho VITROS ® Anti-SARS-CoV-2 IgG.

KEYWORDS SARS-CoV-2; COVID-19; neutralizing antibody; IgG; IgA; RT-PCR; neutralization assay; EIA; CLIA; CMIA.

Since the beginning of the SARS-CoV-2 pandemic, COVID-19 can be seen as an unusual infection. Although it tends to be more serious in certain categories of patients, there is little correlation between clinical course of infection and antibody levels reached after convalescence (1) (2) (3) (4) . Indeed, longitudinal surveys have demonstrated that antibody responses mount earlier in severe COVID patients. However, antibody final titers vary greatly in different patients independently of the clinical course of infection and about 5% patients have undetectable antibody titers despite documented infection (5, 6) . In addition, in patients that do have antibodies, titers tend to decline with time to the point that some of them become seronegative (7) .

Passive antibody therapy is one of the oldest interventions to control infection and is effective against a vast number of pathogens (8) . Effectiveness of this approach relies on the presence of neutralizing antibodies (NAbs) that prevent viral access into susceptible cells by binding specific domains of the viral entry machinery. Indeed, at the top of the SARS CoV2 attachment and fusion protein, the spike (S) protein, there is an N-terminal domain and the SARS CoV2 receptor-binding domain RBD, the part of S responsible for binding the cellular receptor angiotensin-converting enzyme 2 (ACE2); recently, epitope mapping analysis has shown that these regions contain neutralizing epitopes (9).

Passive antibody therapy has shown promising results promising against other betacoronavirus infections, and there is good possibility that COVID 19 convalescent patient plasma (CCP) is also effective against SARS-CoV-2 (10) (11) (12) . In keeping with this hypothesis, infusion of COVID-19 convalescent plasma (CCP) into patients with non-critical disease was found to be beneficial (10, (13) (14) (15) (16) (17) (18) . European Medicine Agency have only approved remdesivir as a specific antiviral drug, but but is associated with a high cost (ema.europa.eu, doi.10.1001/jama.2020.16337). Therefore, clinical trials have been undertaken around the world to evaluate the efficacy of CCP therapy; CCP may ameliorate clinical symptoms when given up to 15-30 days after onset, although many issues including safety remain to be clarified, warranting further studies (19) (20) (21) (22) .

To date, although no definitive scientific evidence supports the adoption of a defined titer of NAbs in donated CCP, the presence of high levels of anti-SARS-CoV-2 NAbs is recommended. This explains why one of the main criteria to qualify for CCP donation in many studies is the presence of NAbs above a certain threshold (23) . To our knowledge, titration of NAbs can only be achieved by the microneutralization test (NT), a biological assay which is labor-intensive, time-consuming, and biohazardous. On the other hand, CCP therapy for hospitalized COVID-19 patients requires screening of thousands of plasma donors. For these reasons, it would be recommended to find simpler, faster and automatable tests allowing screening of large numbers of plasma samples to preselect sera for NT.

In an effort to find an automated serological test that correlates with NT, we have used five European Community-approved diagnostic automated immunoassays that are claimed by manufacturers to quantify, as arbitrary units (AU/ml) or as a signal/cutoff ratio, anti-SARS-CoV-2 antibodies (Abs) and compared these results with NT titers. We found good correlations between the results obtained by NT and 2 of the immunoassays tested, indicating that, under certain conditions, immunoassays can be used to identify donors likely to have high NAb titers.

This study was carried out with patients adhering to clinical trial "TranSfUsion of NT assay. NT was carried out using a limiting dilution assay method established by NeuCoV-NET, an Italian network established by selected laboratories (24) . All CCP used under the TSUNAMI study were certified by laboratories belonging to NeuCoV-NET, who generated a set of proficiency sera sent periodically to all laboratories and used to gauge NT assay performances and calculate inter-laboratory variability.

The NT assay was performed with the SARS-CoV-2 strain SARS-CoV-2/Human/ITA/PAVIA10734/2020. This strain was isolated from a 74 years-old symptomatic male patient isolated in February 2020 that has been fully sequenced (sequence available on GISAID, accession no. EPI_ISL_568579, and GeneBank, accession no. M527178.1). It is of clade G and bears D614G mutation in the S protein (25) . SARS-CoV- Immunoassays. Serum samples were analyzed using the following five commercial serological assays: LIAISON® XL SARS-CoV-2 S1/S2 IgG CLIA kit S1/S2 based antigen (ORT). The antigen target and assay methods are described in Table 1 . Samples were tested unblinded by all assays by senior biomedical technologists within one week from collection.

All testing was performed according to manufacturers' instruction and results were given following the interpretation criteria supplied by the manufacturer. Undetermined results were interpreted as negative for the purpose of the study.

Statistical Analysis. Univariate linear regression was assessed using Pearson's correlation coefficient. Sensitivity and specificity of each test when used to predict NT titers >80 by receiver operating characteristic (ROC) curve analysis was obtained by calculating the area under the curve (AUC), with corresponding 95% Confidence Interval (CI). The AUC allows to determine the percentage of the results that turn positive by a given test for a given threshold. To minimize inter-assay variations, ROC analysis is shown and discussed for the data obtained with 140 sera that were examined with all five immunoassays and NT.

Youden's index, a common summary measure of the ROC curves, was used to identify the best threshold to discriminate plasma samples with NT titers <80 and >80, then we calculated the corresponding sensitivity and specificity, as shown in Table 2 (27) . Specificity and sensitivity were also set at 0.95 to identify the best threshold values ( Table 2) . Statistics and graphics were performed using R software, version 4.0.3, (https://www.R-project.org/) and Illustrator.

Experimental procedure. The features of the diagnostic immunoassays used in the present work are outlined in Table 1 . The 5 tests in question may be evaluated for their ability to answer three different questions: 1) identifying patients with Covid-19 from patients with no infection. This ability was assessed by the manufacturers and specificity and sensitivity are indicated in each assay instruction sheet. In addition, the immunoassays selected have already been extensively evaluated for their clinical sensitivity and specificity by others (28, 29) . For these reasons, we did not evaluate this ability in the present work. 2) Evaluating the association between NT titers with anti SARS-CoV-2 binding Ab titers. 3) Selecting samples eligible for donation from those that were not amongst NT-positive sera. The last 2 properties are addressed in the following paragraphs and results are shown in Fig.1 and 2 and summarized in Table 2 . The NT titer of ≥80 as a cut-off for NT results was selected because it is just one dilution step below 160, and it would increase the possibility of identifying all suitable donors.

We first explored the association between the values of antibody titers found by the 5 different tests and NT titers by univariate regression analysis (Fig.1) . We examined nearly 200 samples from individuals who suffered from COVID-19 at various degrees of severity with NT and one or more immunoassays. The results shown in Fig.1 revealed that, in order, DIA, EUG and ORT were the best test in terms of correlation with the NT results (R 2 =0.34, 0.32 and 0.24, respectively).

Values observed for EUA and ABB were poorly correlated with NT result and therefore less able to predict neutralization activity.

To confirm the correlation between the values obtained by each immunoassay and NT, ROC curves were generated by plotting the number of false positive fraction of samples using the given immunoassay, on the x axis, and the true positives on the y axis. NT was the reference assay, scoring values of NT titer <80 and ≥ 80 negative and positive, respectively. This analysis allows to determine the association between clinical sensitivity (defined as the fraction of sera that turned out as true positives (>80) by the test analyzed to all sera with NT titers > 80) and specificity (fraction of true negatives (<80) by the test analyzed to all sera with NT titers < 80) for every possible cut-off for each test. The trade-off between clinical sensitivity and specificity for every possible cut-off for ABB, EUA, EUG, DIA, and ORT assays in identifying NT titer of 80 or greater is illustrated in Fig. 2 (27) .

The ROC curves obtained are shown in Fig.2 . A minimum value of 0.95 for sensitivity and specificity and ≥80 NT titer as a threshold were arbitrarily selected to compare assays. To this aim, the AUC under each ROC curve was calculated and are shown for each curve in Fig   2. The AUC allows to determine the percentage of results that turn positive by a given immunoassay at a given NT titer threshold. As can be seen, the two assays ABB and EUA were identified as the worst classifiers, with AUC values of 0.74 and 0.75, respectively. This means that the chance of finding a NT titer ≥80 by ABB and EUA was too close to the noninformative value of 0.50, that corresponds to complete lack of correlation between the values of NT and those of the immune assays tested. In this case, the most suitable tests under these parameters were, in order, EUG, ORT and DIA, with an AUC of 0.89, 0.86, 0.77, respectively.

Finding the best assay to detect CCP for donations. With the idea of providing an automated method to identify CCP potentially exploitable for the TSUNAMI study, we derived Youden's index from the ROC curves. Youden's index, a convenient value summarizing the performance of each assay, is a value varying from 0, when the assay yields equal positive results for sera with and without Nabs, to 1, when no false positives or false negatives are detected. Youden's criteria indicated DIA as the best test in terms of maximum specificity of 0.92 whereas maximum balance between specificity and sensitivity was, again, EUG: at a cut-off of 7.3, sensitivity was 0.79 and specificity was 0.88.

Moreover, as shown in Table 2 , when specificity is fixed at 0.95, EUG again proved to be the best test, with a cut-off of 8.5 (S/C) and a sensitivity of 0.65. Instead, when sensitivity was fixed at 0.95, the best test was ORT, at a cut-off of 10.5 (S/C) and a specificity of 0.55.

The development of NAbs among patients with COVID 19 exhibits substantial variability after convalescence, in that some patients develop higher antibody responses compared to others (6) . CCP is being tested to treat patients with severe COVID 19, provided that the NAb titer is ≥ 160. There have been contradictory conclusions in terms of clinical benefits offered by CCP therapy, with some early studies showing benefits (10, 13, 14) while others did not (35) . Because effective treatment options against COVID19 are still few, further investigation on this therapeutic option are warranted.

Because the NT is particularly laborious, a prescreening automated diagnostic method allowing the detection of therapeutic plasma that does not miss high-NT titer samples is urgently needed. Such a reliable immunoassay would allow exclusion of negative samples before the NT test, thereby substantially reducing overall labor and healthcare costs.

Furthermore, given that NAbs decline with time, the preselection of sera by an automated assay would facilitate prompt enrolment of convalescent donors (7, (25) (26) (27) (28) (29) .

In this study, we tested five widely used diagnostic assays to assess which would be the most predictive of a NT titer ≥80, in order to obtain therapeutic CCP donations to treat COVID-19 patients. Except for ABB, the remaining four tests target the viral receptor S protein as a whole or as one or both subunits. ABB uses the nucleocapsid protein as its target antigen, Abs against which appear first and in higher amounts compared to S protein (2, 3) . This test is used to identify SARS-CoV-2 seropositive donors by the Italian blood bank service. It was not surprising, therefore, that ABB did not correlate well with NT titers and with a very low specificity. Poor performances were also observed with EUA, which detects IgA anti-S1 protein, and showed very low specificity in detecting sera with NT titers >80. Although IgA exhibit neutralizing activity and are known to play a protective role against respiratory viruses at a mucosal level (36, 37) , it is likely that serum IgA levels are very low and do not significantly contribute to serum neutralizing activity.

DIA, EUG and ORT performed satisfactorily, as they showed sufficient sensitivity and specificity. In agreement with a previous study (38) , the assay which correlated best with NT and showed a good balance between sensitivity and specificity was EUG.

As for the TSUNAMI study, when the demand for CCP is limited, it is important to minimize false positives in the pre-screening procedure, even if this implies performing more NT assays. In this scenario, we suggest to fix specificity at 0.95 and, consequently, ORT is considered to be most appropriate. Conversely, in the setting of high demand for CCP (e.g., if CCP becomes an established therapy) elevated sensitivity (0.95) is preferable to minimize donor loss. In these conditions, EUG and ORT would be considered the best assays.

All samples were thawed, analyzed in parallel and in a narrow time interval to minimize inter-assay variability or inconsistent readings due to sample degradation. However, they were collected at different times from recovery, therefore our data cannot be useful in determining the percentage of recovered patients with neutralizing Abs because Nabs tend to decrease rapidly with time from recovery (32) .

From these results, we have determined the performance of the assays by maximizing detection of positive samples, e.g., by setting the sensitivity to 0.95 that allows the detection of 95% positive samples, or by detecting only true positive samples, e.g., by setting specificity to 0.95 to reduce spurious positive samples to 5%. By using these conditions, the best assay was ORT followed by ABB. Conversely, by setting specificity to 0.95, the best assay was EUG followed by ORT (Table 2 ). In addition, this analysis demonstrated that Youden's criteria privileged either sensitivity (i.e., higher ratio of false positive results, ABB and EUG) or specificity (i.e., higher ratio of false negative results, DIA, EUG, and ORT) ( Figure 2 , Table 2 ).

A possible limitation of the study presented is the limited number of samples taken into consideration. However, the results are consistent with those of Harvala et al., who also found EUG suitable to detect high-NT-titer sera (23) . In addition, they confirm those of Padoan et al, who also found ORT as the most reliable immunoassay to detect high NT titers in sera (28, 38 (40) . Therefore, our results might useful for post-vaccination measurement of protective immunity, since the vaccines available at present employ S protein sequences from viral strains isolated in 2019.

To conclude, we believe that this study provides clinicians with important information to aid in the selection of donors for CCP therapy. Cut-off values of the serological assays either as obtained following Youden's criteria (1) or by setting sensitivity (2) or specificity (3) ≥0.95. CI for specificity and sensitivities were 95%.

Mechanisms of SARS-CoV-2 Transmission and Pathogenesis

The immunology of SARS-CoV-2 infections and vaccines

Overview of Immune Response During SARS-CoV-2 Infection: Lessons From the Past

Host-pathogen interaction in COVID-19: Pathogenesis, potential therapeutics and vaccination strategies

Diagnostic accuracy of serological tests and kinetics of severe acute respiratory syndrome coronavirus 2 antibody: A systematic review and meta-analysis

The complexities of SARS-CoV-2 serology

Decline of Humoral Responses against SARS-CoV-2 Spike in Convalescent Individuals

Return to the past: the case for antibody-based therapies in infectious diseases

Potent neutralizing antibodies against multiple epitopes on SARS-CoV-2 spike

Treatment of 5 Critically Ill Patients With COVID-19 With Convalescent Plasma

SARS-CoV-2 neutralizing antibody structures inform therapeutic strategies

Effects of human anti-spike protein receptor binding domain antibodies on severe acute respiratory syndrome coronavirus neutralization escape and fitness

Effect of Convalescent Plasma Therapy on Time to Clinical Improvement in Patients With Severe and Life-threatening COVID-19: A Randomized Clinical Trial

Effectiveness of convalescent plasma therapy in severe COVID-19 patients

Neutralizing and cross-reacting antibodies: implications for immunotherapy and SARS-CoV-2 vaccine development

SARS-CoV-2 convalescent plasma in severely ill adults with COVID-19: A structured summary of a study protocol for a randomized controlled trial

Convalescent plasma or hyperimmune immunoglobulin for people with COVID-19: a living systematic review

Treatment for emerging viruses: Convalescent plasma and COVID-19

Convalescent Plasma to Treat COVID-19: Possibilities and Challenges

Operational protocol for donation of anti-COVID-19 convalescent plasma in Italy

Prevalence of SARS-CoV-2 specific neutralising antibodies in blood donors from the

SARS-CoV-2 Phylogenetic Analysis

Development of a low-cost, simple test for routine detection of Mycoplasma contamination in cell cultures

Youden Index and optimal cut-point estimated from observations affected by a lower limit of detection

Analytical and clinical performances of five immunoassays for the detection of SARS-CoV-2 antibodies in comparison with neutralization activity

Evaluation of Three Commercial SARS-CoV-2 Serologic Assays and Their Performance in Two-Test Algorithms

Persisting Neutralizing Activity to SARS-CoV-2 over Months in Sera of COVID-19 Patients

A systematic review of antibody mediated immunity to coronaviruses: kinetics, correlates of protection, and association with severity

Antibody responses to SARS-CoV-2 in patients with COVID-19

Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Responses in Coronavirus Disease Patients

Review of Current Advances in Serologic Testing for COVID-19

Convalescent plasma in the management of moderate covid-19 in adults in India: open label phase II multicentre randomised controlled trial (PLACID Trial)

Induction and Subversion of Human Protective Immunity: Contrasting Influenza and Respiratory Syncytial Virus

Mucosal Immunity in COVID-19: A Neglected but Critical Aspect of SARS-CoV-2 Infection

Convalescent plasma therapy for the treatment of patients with COVID-19: Assessment of methods available for antibody detection and their correlation with neutralising antibody levels

351 and B.1.1.7 to Antibody Neutralization. bioRxiv

1.1.7 pseudovirus by BNT162b2 vaccine-elicited human sera

We are grateful to the members of the TSUNAMI study for providing blood samples and to Dr. C. Egan for revision. 

The authors declare no competing financial interest.

Youden's index, and 95% sensitivity and specificity. Sensitivity set at 95% allows the detection of 95% positive samples, the specificity set at 95% limits detection of negative samples to 5%.