key: cord-0812346-0ps2pxz6 authors: Saker, Kahina; Pozzetto, Bruno; Escuret, Vanessa; Pitiot, Virginie; Massardier-Pilonchéry, Amélie; Mokdad, Bouchra; Langlois-Jacques, Carole; Rabilloud, Muriel; Alfaiate, Dulce; Guibert, Nicolas; Fassier, Jean-Baptiste; Bal, Antonin; Trouillet-Assant, Sophie; Trabaud, Mary-Anne title: Evaluation of Commercial Anti-SARS-CoV-2 Neutralizing Antibody Assays in seropositive subjects date: 2022-04-27 journal: J Clin Virol DOI: 10.1016/j.jcv.2022.105169 sha: b5f75b06ef4cf7dbd75ecffcdfbc7801e9f1e10e doc_id: 812346 cord_uid: 0ps2pxz6 The virus neutralization test (VNT) is the reference for the assessment of the functional ability of neutralizing antibodies (NAb) to block SARS-CoV-2 entry into cells. New competitive immunoassays measuring antibodies preventing interaction between the spike protein and its cellular receptor are proposed as surrogate VNT (sVNT). We tested three commercial sVNT (a qualitative immunochromatographic test and two quantitative immunoassays named YHLO and TECO) together with a conventional anti-spike IgG assay (bioMérieux) in comparison with an in-house plaque reduction neutralization test (PRNT(50)) using the original 19A strain and different variants of concern (VOC), on a panel of 306 sera from naturally-infected or vaccinated patients. The qualitative test was rapidly discarded because of poor sensitivity and specificity. Areas under the curve of YHLO and TECO assays were, respectively, 85.83 and 84.07 (p-value >0.05) using a positivity threshold of 20 for PRNT(50), and 95.63 and 90.35 (p-value =0.02) using a threshold of 80. However, the performances of YHLO and bioMérieux were very close for both thresholds, demonstrating the absence of added value of sVNT compared to a conventional assay for the evaluation of the presence of NAb in seropositive subjects. In addition, the PRNT(50) assay showed a reduction of NAb titers towards different VOC in comparison to the 19A strain that could not be appreciated by the commercial tests. Despite the good correlation between the anti-spike antibody titer and the titer of NAb by PRNT(50), our results highlight the difficulty to distinguish true NAb among the anti-RBD antibodies with commercial user-friendly immunoassays.  Performance of surrogate virus neutralization tests in seropositive samples  Competitive immunoassays based on the interaction between spike protein and ACE2  Comparison with plaque reduction neutralization test considered as gold standard  Comparison with a classical serological assay detecting anti-RBD IgG  Surrogate virus neutralization tests can't predict neutralization of variant with an in-house plaque reduction neutralization test (PRNT 50 ) using the original 19A strain and different variants of concern (VOC), on a panel of 306 sera from naturally-infected or vaccinated patients. The qualitative test was rapidly discarded because of poor sensitivity and specificity. Areas under the curve of YHLO and TECO assays were, respectively, 85.83 and 84.07 (p-value >0.05) using a positivity threshold of 20 for PRNT 50 , and 95.63 and 90.35 (pvalue =0.02) using a threshold of 80. However, the performances of YHLO and bioMérieux were very close for both thresholds, demonstrating the absence of added value of sVNT compared to a conventional assay for the evaluation of the presence of NAb in seropositive subjects. In addition, the PRNT 50 assay showed a reduction of NAb titers towards different VOC in comparison to the 19A strain that could not be appreciated by the commercial tests. Coronavirus disease 2019 (COVID- 19) is an emerging disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and since late 2020, vaccines against SARS-CoV-2 have been available worldwide. In recent months, a large number of commercial immunoassays have been developed for the detection of specific anti-SARS-CoV-2 antibodies (1, 2) . However, the presence of anti-SARS-CoV-2 antibodies does not indicate whether the antibodies are able to neutralize the virus that has been reported to have a role in the protection from COVID-19 both in animals and humans (3) . The gold standard for assessing the ability of antibodies to prevent the virus from entering into susceptible cells is the virus neutralization test (VNT) (4), but it requires a biosafety level 3 laboratory and takes approximately 10 days to complete. This has led to the development of SARS-CoV-2 surrogate virus neutralization tests (sVNT) that are more simple and rapid; these are based on the competition between patient antibodies and the angiotensin converting enzyme 2 (ACE2) receptor protein for binding to the spike receptor binding domain (RBD) that mediates the entry of the virus into susceptible cells (5) . These competitive immunoassays, which can be conducted using qualitative immunochromatographic cassettes or quantitative automated or manual enzyme-linked immunosorbent assay (ELISA) platforms, allow rapid and easy processing of large numbers of samples in conventional serological laboratories. However, the performance of these newly developed commercial sVNT assays by comparison to classical serological assays detecting anti-RBD IgG and/or to the reference plaque reduction neutralization test 50% (PRNT 50 ) performed with live virus has been poorly evaluated up to now (6) (7) (8) (9) . Moreover, previous studies have evaluated the specificity using seronegative and/or prepandemic serum which do not inform if commercial sVNT can differentiate serum with or without neutralizing antibody in seropositive samples. The aim of the present study was to evaluate the performance of three commercial sVNT and of a classic anti-RBD IgG assay by comparison to NAb titers measured by a conventional PRNT 50 with the original strain (clade 19A) and various clades in seropositive samples. This prospective longitudinal cohort study was conducted at the laboratory associated with the national reference center for respiratory viruses (university hospital of Lyon, France). Table 1 . For the present study suppliers kindly provided all serological kits used; there were 81 Dynamiker tests available, and a sufficient number of YHLO, TECO, and bioMérieux kits for all samples tested herein (Supplementary figure S1). YHLO and TECO are quantitative assays, bioMérieux is a semi-quantitative assay and Dynamiker is a qualitative assay. Each assay was compared to the VNT (PRNT 50 ); the latter was used for the detection and titration of neutralizing antibodies, as previously described (4; see supplementary materials and methods). A threshold of 20 and of 80 was used (PRNT 50 ≥ 20/80); the threshold of 20 of the live virus neutralization assay was considered as the detection limit of this assay. Thus, samples with PRNT 50 titers below 20 are considered as negative for the presence of neutralizing antibodies. In contrast, the PRNT 50 threshold of 80 was a cutoff value assumed to discriminate high from low NAb titer. First of all, we compared performance of sVNT assays with VNT (clade 19A) on 81 and 246 convalescent samples for the three sVNT and the two quantitative sVNT respectively. As the YHLO assay provided the best results among the investigated sVNT, the added-value of this test compared to a commercial serological assay detecting anti-RBD IgG (bioMérieux) was The correlation between Ab concentrations obtained by each assay was investigated using Pearson correlation coefficients and 95% confidence interval [95%CI] . Receiver operating characteristic curves (ROC) were built to estimate the performance of YHLO, TECO, and bioMérieux assays for the detection of the presence of neutralizing antibodies considering the VNT considered as gold standard. Area under the curves (AUC) were compared using the Delong's test. For the YHLO and TECO assays the positive threshold according to the manufacturers' instructions were found to be too low; these were recalculated using the Youden index. Results of various clades were represented with ellipses that show the 95% CIs for different clades schedules, assuming multivariate normal distributions. Statistical analyses were conducted using GraphPad Prism ® software (version 8; GraphPad software, La Jolla, CA, USA) and R software, version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria). A p-value <0.05 was considered statistically significant. Written informed consent was obtained from all participants; ethics approval was obtained from the regional review board for biomedical research in April 2020 (Comité de Protection des Personnes Sud Méditerranée I, Marseille, France; ID RCB 2020-A00932-37), and the study was registered on ClinicalTrials.gov (NCT04341142) (10) . According to the VNT (PRNT 50 ) with clade 19A and the positive threshold of ≥ 20, neutralizing antibodies were found in 54.5% (134/246) in the convalescent cohort and 100% (60/60) in the vaccinated cohort; using the positive threshold of ≥ 80 this was the case for 10,6% (26/246) and 95% (57/60), respectively. The performance of sVNT assays was estimated among the 81 samples for which data was available using both VNT thresholds [PRNT 50 ≥ 20 (59/81) and PRNT 50 ≥ 80 (24/81)] and according to the positive threshold indicated by the corresponding manufacturer. The Dynamiker qualitative test exhibited very weak performance both in terms of sensitivity and specificity, and was discarded from the following steps of the evaluation ( Figure 1 , Table 2 ). The combination of a good AUC but a very low specificity observed with the positive threshold indicated by the manufacturers led us to determine the best-fit cut-offs for the YHLO and TECO assays using the Youden index. Table 2) . The correlation coefficient (ρ [IQR]) between the YHLO assay and the VNT (PRNT 50 ) was Figure 2D ). Regarding the neutralizing capacity of serum against VOCs, the median fold-reduction in Nab titers varied between 1.3 against alpha strain and 2.7 against beta strain in comparison to 19A strain ( Figure 3A and 3B Despite good correlation between concentrations of anti RBD IgG detected with YHLO or bioMérieux assays and neutralizing antibodies titers against each variant, the same titer of binding Abs overestimates titers of variant Nabs which are lower against the variants than the wild type. The performance of the qualitative Dynamiker assay was found to be poor, both in terms of sensitivity and in terms of specificity. The two other quantitative sVNT assays evaluated in the present study were found to be more sensitive but their specificity was extremely low since, at the manufacturers' cutoff, most samples (TECO assay) or all of them (YHLO assay) from convalescent individuals with no detectable NAb using the live virus neutralization assay were positive for NAb with sVNT. It can be postulated that part of the antibodies detected by these ELISA are able to interfere with the interaction between ACE receptor and the viral RBD but not to prevent cell entry of the virus; this may be related to the affinity/avidity of antibodies that is reported to be low after primary infection or first vaccine dose (11, 12) , and is likely to be even more the case for the population included herein who were sampled 6 months after infection. Despite this low specificity, these assays correlated with the live virus neutralization assay as also found in other studies (5, 6, (13) (14) (15) . The low specificity could also be attributed to a lack of sensitivity of live VNT but it is rather unlikely that decreasing the threshold below 20 would be clinically relevant, such low titer having little chance to be protective in vivo (16) . In addition, the manufacturer positivity threshold was determined using pre-pandemic serum without any anti SARS-CoV-2 antibodies. Nevertheless, the value of sVNT use is to distinguish antibody with or without neutralizing capacity, and we designed our study to establish this performance using only seropositive samples without presence of Nab to establish specificity for sVNT assays which can explained the low specificity observed in our study. It seems thus preferable to increase the sVNT cutoff to improve specificity, and go closer to the protective threshold. Our data from ROC curves would indicate that, for the detection of NAb, a threshold of 70 IU/ml and 30 AU/ml should be applied for the TECO an YHLO assays, respectively. However, these data have been obtained from infected subjects late after infection, at a time where antibodies are decreasing (17) . This could explain the low frequency of sera with detectable NAb using the VNT, and also the discrepancy in terms of specificity between our results and previous ones using samples earlier after infection (18) (19) (20) (21) . With time, waning of antibodies could have more impact on the blocking of infection than interference with ACE binding. The study of Von Rein et al (22) suggested that the correlation between sVNT and VNT was greater at higher level of neutralization titer and they concluded that sVNT are only useful when inhibition was above 50%, which is more consistent with our data. Most of the previous studies used the cPASS assay from GeneSript,showing the correlation of competitive immunoassay to live VNT, with good sensitivity and specificity compared to VNT (5, 6, 9, 14, 15, 19, 20, 22, 23) . Only a few studies reported results with the TECO (15, 20) or YHLO assays (7, 8, 21) . Of note the study of Chan et al (21) found a diagnostic cutoff, with the YHLO assay, of 27,7 AU/ml, which is closed from ours of 30 AU/ml. Other studies have compared VNT and sVNT with assays detecting IgG binding to RBD or S proteins and observed a correlation between them (6, 8, 9) . Fisher et al. (6) found that the correlation between sVNT and antibody binding assay is better for samples with high than low PRNT. Others studies have shown that the correlation between VNT and antibody binding assays was lower than between sVNT and antibody binding assays (9) . It remains that high-throughput live virus neutralizing assays is not possible, and for this binding or competitive antibody immunoassays could be used but caution should be taken when interpreting the result, regardless of the assay used. Despite the high performance, based on the AUC of the ROC curve, of the two competitive automated immunoassays evaluated in our study, taking VNT as gold standard, we did not demonstrate any added value of sVNT compared to serological assay detecting anti-RBD IgG for evaluating the presence of Nab in seropositive subjects. These results highlight the difficulty to distinguish the Nab among anti-RBD IgG using a standard immunoassay. This difficulty could be further extrapolated considering the antibodies able to neutralize the SARS-CoV-2 variants. Using serum collected one month post full vaccination in patients with high Nab titers, we confirmed the diminution of Nab titers against different SARS-CoV-2 variant compared to initial strain. Nevertheless, the RBD coated in these competitive sVNT is not adapted to virus evolution and are not able to detect the decrease of NAb titers. To date, VNT remains the only way to detect Nabs against VOC. Taking together, from our data and those previously published, the predictive value of surrogate neutralization assays is still not obvious in all population (infected and/or vaccinated, after priming or boost immunization, early versus late after immunization). In addition, sVNT are not able to predict neutralization of variant, and thus improvements are needed before they can be considered equivalent to VNT to detect NAbs able to protect from infection. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The respective suppliers kindly provided all the serological kits used in the present study. Lyon. The respective suppliers kindly provided all the serological kits used. We thank the staff of the occupational health and medicine department of the Hospices Civils de Lyon who No statistically significant difference was observed between the two tests for both thresholds. Positivity was established according to manufacturers' instructions. Sensitivity and specificity data were those described in the instruction for utilization sheet from each manufacturer. Specificity given by manufacturers was obtained from pre pandemic samples. Evaluation of High-Throughput SARS-CoV-2 Serological Assays in a Longitudinal Cohort of Patients with Mild COVID-19: Clinical Sensitivity, Specificity, and Association with Virus Neutralization Test Evaluation of Commercial Anti-SARS-CoV-2 Antibody Assays and Comparison of Standardized Titers in Vaccinated Health Care Workers Isolation of potent SARS-CoV-2 neutralizing antibodies and protection from disease in a small animal model A longitudinal study of SARS-CoV-2-infected patients reveals a high correlation between neutralizing antibodies and COVID-19 severity Structure of the SARS-CoV-2 spike receptorbinding domain bound to the ACE2 receptor A combined strategy to detect plasma samples reliably with high anti-SARS-CoV-2 neutralizing antibody titers in routine laboratories Neutralizing Antibodies in COVID-19 Patients and Vaccine Recipients after Two Doses of BNT162b2. Viruses Dynamics of Neutralizing Antibody Responses Following Natural SARS-CoV-2 Infection and Correlation with Commercial Serologic Tests. A Reappraisal and Indirect Comparison with Vaccinated Subjects. Viruses Evaluation of a Surrogate Enzyme-Linked Immunosorbent Assay-Based Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) cPass Neutralization Antibody Detection Assay and Correlation With Immunoglobulin G Commercial Serology Assays Assessment of serological techniques for screening patients for COVID-19 (COVID-SER): a prospective, multicentric study Antibody and B cell responses to SARS-CoV-2 infection and vaccination. Cell Host Microbe mRNA vaccination of naive and COVID-19-recovered individuals elicits potent memory B cells that recognize SARS-CoV-2 variants Serological anti-SARS-CoV-2 neutralizing antibodies association to live virus neutralizing test titers in COVID-19 paucisymptomatic/symptomatic patients and vaccinated subjects A SARS-CoV-2 surrogate virus neutralization test based on antibody-mediated blockage of ACE2-spike protein-protein interaction Two novel SARS-CoV-2 surrogate virus neutralization assays are suitable for assessing successful immunization with mRNA-1273 Correlates of protection against symptomatic and asymptomatic SARS-CoV-2 infection Sera Neutralizing Activities Against Severe Acute Respiratory Syndrome Coronavirus 2 and Multiple Variants 6 Months After Hospitalization for Coronavirus Disease Correlation of sampleto-cut-off ratio of anti-SARS-CoV-2 IgG antibody chemiluminescent assay with neutralization activity: a prospective multi-centric study in India Evaluation of a SARS-CoV-2 Surrogate Virus Neutralization Test for Detection of Antibody in Human, Canine, Cat, and Hamster Sera Utility of Different Surrogate Enzyme-Linked Immunosorbent Assays (sELISAs) for Detection of SARS-CoV-2 Neutralizing Antibodies Performance of a Surrogate SARS-CoV-2-Neutralizing Antibody Assay in Natural Infection and Vaccination Samples. Diagnostics (Basel) Comparison of potency assays to assess SARS-CoV-2 neutralizing antibody capacity in COVID-19 convalescent plasma A New SARS-CoV-2 Dual-Purpose Serology Test: Highly Accurate Infection Tracing and Neutralizing Antibody Response Detection ROC curves were built to estimate the performance of YHLO (in grey) and TECO (in black) assays for detecting the presence of neutralizing antibodies (PRNT 50 ≥20 (A)) and high neutralizing antibody titre PRNT 50 ≥20 (n=246) AUC, % PRNT 50 ≥80 (n=246) AUC, % IQR: interquartile range, CI: confidence interval, AU: arbitrary unit, UI: unit international