key: cord-0717557-ibswd00a authors: Chen, Hui; Yu, Wanwan; Gao, Xiaojiao; Jiang, Weijun; Li, Xiaojun; Liu, Guorui; Yang, Yang title: A method comparison of three immunoassays for detection of neutralizing antibodies against SARS‐CoV‐2 receptor‐binding domain in individuals with adenovirus type‐5‐vectored COVID‐19 vaccination date: 2022-02-23 journal: J Clin Lab Anal DOI: 10.1002/jcla.24306 sha: a6d3800d28d3b4e4438f7a70d505402abd78d5ca doc_id: 717557 cord_uid: ibswd00a OBJECTIVE: Detecting neutralizing antibodies targeting receptor‐binding domain (RBD) is important for the assessment of humoral protection and vaccine efficacy after vaccination. We compared the performance of three surrogate immunoassays for detection of neutralizing antibodies targeting RBD. METHODS: We analyzed 115 serum samples obtained from individuals with Ad5‐vectored COVID‐19 vaccination using two competitive enzyme‐linked immunosorbent assays (Wantai BioPharm and Synthgene Medical Technology) and one competitive chemiluminescence assay (YHLO Biotech). Performance evaluation and methodology comparison were performed according to the Clinical and Laboratory Standards Institute related guidelines. RESULTS: The precision met the manufacturers’ statements. The linear range of the WANTAI was 0.0625–0.545 U/ml and the YHLO was 0.260–242.4 U/ml. The WANTAI’s limit of blank (LoB) and limit of detection (LoD) were 0.03 and 0.06 U/ml, respectively. The YHLO’s LoB and LoD were 0.048 and 0.211 U/ml, respectively. The correlations of semi‐quantitative results of Synthgene with quantitative results of YHLO (ρ = 0.566) and WANTAI (ρ = 0.512) were medium. For YHLO and WANTAI, there was a good agreement (0.62) and a strong correlation (ρ = 0.931). Passing–Bablok analysis and Bland‐Altman plot showed a positive bias (112.3%) of the YHLO compared to the WANTAI. The exclusion of samples >50 U/ml did not decrease bias. CONCLUSION: These findings contribute to a deeper understanding of surrogate viral neutralization assays and provide useful data for future comparison studies. The outbreaks of coronavirus disease 2019 hit the world health, economy and society severely. 1 Since June 2021, severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) has caused more than 100 million infections and 3.6 million deaths. 2 Vaccines are one of the most effective approaches to prevent viral infection. 3 Nowadays, more than 200 vaccines for SARS-CoV-2 are being developed or in clinical trials. At least five vaccines including adenovirus-vectored vaccine, inactivated virus vaccine, and recombinant proteins vaccine have been approved for clinical use by the Chinese Food and Drug Administration. 4 Serological testing played an important role in assessment of immunity in the vaccinated populations. 5 Most SARS-CoV-2 Ab detection assays are based on nanoparticle-based lateral-flow test (GNT) strip, enzyme-linked immunosorbent assay (ELISA), chemiluminescence assay (CLIA), and electrochemiluminescence immunoassay (ECLIA). 6, 7 The common antigens used as the target were spike (S) and nucleocapsid (N) due to the high immunogenicity. 8 Anti-RBD antibodies, which are produced after vaccination, acted as the main neutralizing antibodies by blocking virus binding to the host angiotensin-converting enzyme 2 (ACE2). The neutralizing antibodies targeting RBD levels have been used to evaluating humoral immune response following COVID-19 vaccination. Nowadays, the serological surrogate immunoassays are being developed for the neutralizing antibodies targeting RBD. [9] [10] [11] [12] [13] These surrogate viral neutralization assays (sVNTs) based on the same principle of competitive binding. [14] [15] [16] Utilizing purified receptor-binding domain from S protein and ACE2 receptor, these assays enable specific antibodies to block RBD binding to ACE2. Qualitative or quantitative determination could be achieved by immunolabeling. As the number of sVNTs is growing; however, the performances are not well-known. In this study, we compared the performances of three surrogate immunoassays including two competitive ELISA assay and one competitive CLIA assay for detection of the neutralizing antibodies targeting RBD in serums from vaccinated individuals. All three assays have received the mark on Conformité Européene. Methodology comparison and bias estimation were conducted. From January 2021 through May 2021, 115 participants (78 males and 37 females; age 42.0 ± 7.5 years, range 20-68) who received one injection of recombinant adenovirus type-5-vectored COVID-19 vaccine were well informed and enrolled in this study. Inclusion criteria and exclusion criteria of vaccination were as described 17 (Table 1) . Serum samples were collected 4 weeks after vaccination. Isolation of serum was achieved by centrifuging at 1700 g for 10 min. For the WANTAI ELISA assay, calibration curve was performed according to the manufacturers' instructions and fitted using four-parametric logistic curves. For the CLIA assay, standard curve was re-conducted by using a two-fold serial dilution of 16 U/ml Wantai's kit standard to convert arbitrary units per millilitre (AU/ml) into units per millilitre (U/ml). The standard used is calibrated against NIBSC 20/136 stand-ard18. The concentration of one Wantai unit (U/ml) is ≈25.7 IU/ml (NIBSC 20/136). Data were analyzed and plotted with GraphPad Prism 8.0 (GraphPad Software, Inc.). If the concentration of the SARS-CoV-2 neutralizing antibody targeting RBD in specimen exceeded the linear range, it is necessary to properly dilute the specimen with diluent. The precision was evaluated using serum samples has analyte values near the concentrations the manufacturer used to establish the pre- Linearity assessment for the two quantitative assays was performed as described in the CLSI EP6-A guideline. The sera sample with high according to y = ax ± b, where y was the measured concentration and x was the expected concentration. When a ranged from 0.97 to 1.03 and R 2 was more than 0.95, with b closer to zero, it could be assumed that measurements were in the linear range. The limit of blank (LoB) and limit of detection (LoD) were determined according to the EP17-A2 protocol. The initial LoB estimate was achieved with direct measurement of the zero-level sample diluent (n = 20). Then, the desired concentration range of low-level samples was identified as 1-5 times the initial estimated LoB. Five blank samples and five low-level samples were detected by the two quantitative assays in four replicates over 3 days (n = 60). For the LoB assessment, the nonparametric analysis was used. The LoB estimate was calculated using the formula: LoB = X 57 + 0.5 × (X 57 − X 58 ). For the LoD evaluation, the precision profile approach was adopted. The LoD was then calculated: LoD = LoB + 1.645 SD, where SD was estimated by the distribution of values measured in the serum pools with very low levels. Methodology comparison and bias estimation were performed according to the CLSI EP9-A3 protocol. For comparison, method data were analyzed and displayed. Statistical analysis was carried out with SPSS version 22.0 (IBM) and MedCalc Software (Mariakerke). The overall, positive, negative percent agreement and Cohen's κ coefficient were calculated to demonstrate the concordance between the three assays. κ values less than 0.40 mean poor agreement, those between 0.40 and 0.60 mean moderate agreement, those between 0.60 and 0.80 mean good agreement and those over 0.80 mean excellent agreement. 19 Experimental data were analyzed using t-test. Spearman's correlation coefficient was used as an assessment of the correlation between detected results of the three assays. A correlation coefficient was categorized as follows: |r| < 0.2, poor; 0.2 ≤ |r| < 0.4, weak; 0.4 ≤ |r| < 0.6, moderate; 0.6 ≤ |r| < 0.8, strong; 0.8 ≥ |r|, excellent. 20 Pairwise concurrence of the two quantitative assays was attained using Passing-Bablok regression models and Bland-Altman plots. To quantitative the levels of the neutralizing antibodies targeting RBD in clinical samples, we first calculated standard curves of WANTAI and YHLO. The R 2 of WANTAI and YHLO was 0.9999 and 0.9905, respectively. The standard curves were presented in Figure 1A ,B. As the ELISA from Synthgene were qualitative experiment, the results of Synthgene assay were only represented as negative or positive. Synthgene, and YHLO were illustrated in Table 3 . The data obtained showed satisfactory precision for the low, medium, and high levels, which were lower than those claimed by the manufacturers. The WANTAI ELISA assay is reported to be linear in 0.0625-0.5 U/ ml without sample dilution by the manufacturers. So, we mixed high-level at 0.6 U/ml with low-level serum pools at 0.0625 U/ml in various ratios. The WANTAI showed a good linear range between 0.0625 and 0.545 U/ml (R 2 = 0.9966, Figure 2A ). Since no linearity information is available on YHLO, a wide range of values of tested mixes were prepared. The YHLO anti-RBD-specific antibody did not deviate from linearity in the entire tested range (0.260-242.4 U/ml) (R 2 = 0.9993, Figure 2B ). The LoB and the corresponding LoD for WANTAI, calculated as previously described, were 0.03 and 0.06 U/ml. The LoB and the corresponding LoD for YHLO were estimated to be 0.048 and 0.211 U/ml. The detection results obtained by the WANTAI ELISA, Synthgene ELISA, and YHLO CLIA were reported in Table 4 and Figure 3 . For Synthgene and WANTAI, the negative percent agreement (NPA) was Figure 3B ). The concordances between these two quantitative assays were shown in Figure 4A ). In Bland-Altman plot, the YHLO showed a positive mean bias% from the WANTAI, which was found to be 112.3% higher. Moreover, the 95% limits of agreement was very wide as shown in Figure 4B , emphasizing a poor concurrence. To our surprise, the exclusion of samples >50 U/ml did not decrease the bias, but large the bias (121.4%), with a slope of 4.5604 (95% CI, 4.0144- 5.1969) and intercept of −0.3255 (95% CI, −1.8121 to 0.7019) ( Figure 4C,D) . The neutralizing antibody targeting RBD level is a highly significant indicator for vaccine efficacy. Nowadays, many efforts have been devoted to developing high-throughput assays for detection of SARS- CoV-2 neutralizing antibody which can allow for use in routine clinical laboratory. In this study, we introduced sVNTs targeting neutralizing antibodies against RBD and reported their different performances. Of the three assays included in our study, the competitive ELISA assay manufactured by Wantai BioPharm has been used to identify (R 2 = 0.9256). 15 The YHLO CLIA seemed to be a better one among the surrogate viral neutralizing assays that have been reported. First, the Synthgene ELISA kit is a newly developed kit, which can only provide qualitative or semi-quantitative results in our study. As shown, this assay had the highest detection rate among the three assays. Our study could suggest that this assay is sufficiently sensi- Additionally, we would also like to highlight that, the neutralizing antibodies targeting RBD is not the only indicator of seroconversion after vaccination. Antibody against the N-terminal domain (NTD) also involved in immune response. 29, 30 Cellular immunity, are an equally important component as humoral immunity in acquiring immunity. 9 The vaccines immune responses varies depending on the vaccine types, the compositions, and vaccination routes. 31 The increasing novel SARS-CoV-2 variants present new insights. 32 Although serologic tests could never fully substitute virus neutralization test, the above newly assays described are of practical utility to improve test capacity and long-term monitor neutralizing antibody levels, facilitating large-scale vaccine evaluation. In conclusion, we found good agreement and correlation among three commercial immunoassays. The instrumented YHLO CLIA assay has a broader linear range and take advantage of simplicity and efficiency over manually ELISAs. Inconsistent results between the CLIA and ELISA indicated that they are not interchangeable in the determination of neutralizing antibodies targeting RBD. Our study contributed to a deeper understanding of these three sVNTs. We would like to thank all participants for their help in this study. There is no conflict of interests. The primary data that support the findings of this study are available from the corresponding author upon a reasonable request. Xiaojun Li https://orcid.org/0000-0003-2396-7188 A novel coronavirus outbreak of global health concern Rapid COVID-19 vaccine development COVID-19: vaccine's progress Inference of SARS-CoV-2 spike-binding neutralizing antibody titers in sera from hospitalized COVID-19 patients by using commercial enzyme and chemiluminescent immunoassays IFCC interim guidelines on serological testing of antibodies against SARS-CoV-2 Update on serologic testing in COVID-19 Analytical and clinical performances of a SARS-CoV-2 S-RBD IgG assay: comparison with neutralization titers Detection of SARS-CoV-2-specific humoral and cellular immunity in COVID-19 convalescent individuals Kinetics of viral load and antibody response in relation to COVID-19 severity Comparison of two commercial surrogate ELISAs to detect a neutralising antibody response to SARS-CoV-2 Diagnostic efficiency of three fully automated serology assays and their correlation with a novel surrogate virus neutralization test in symptomatic and asymptomatic SARS-COV-2 individuals Performance evaluation of three automated quantitative immunoassays and their correlation with a surrogate virus neutralization test in coronavirus disease 19 patients and pre-pandemic controls Validation of a commercially available indirect assay for SARS-CoV-2 neutralising antibodies using a pseudotyped virus assay Heterologous primeboost vaccination with ChAdOx1 nCoV-19 and BNT162b2 A SARS-CoV-2 surrogate virus neutralization test based on antibody-mediated blockage of ACE2-spike protein-protein interaction Evaluation of a SARS-CoV-2 surrogate virus neutralization test for detection of antibody in human, canine, cat, and hamster sera Harmonization of six quantitative SARS-CoV-2 serological assays using sera of vaccinated subjects The kappa statistic in rehabilitation research: an examination Straightforward Statistics for the Behavioral Sciences Immunogenicity and safety of a recombinant adenovirus type-5-vectored COVID-19 vaccine in healthy adults aged 18 years or older: a randomised, double-blind, placebo-controlled, phase 2 trial Safety, tolerability, and immunogenicity of a recombinant adenovirus type-5 vectored COVID-19 vaccine: a dose-escalation, open-label, non-randomised, first-inhuman trial A longitudinal study of SARS-CoV-2-infected patients reveals a high correlation between neutralizing antibodies and COVID-19 severity Comparative performance of five commercially available serologic assays to detect antibodies to SARS-CoV-2 and identify individuals with high neutralizing titers Antibody detection and dynamic characteristics in patients with coronavirus disease 2019 Diagnostic accuracy of serological tests for COVID-19: systematic review and metaanalysis Evaluation of two chemiluminescent and three ELISA immunoassays for the detection of SARS-CoV-2 IgG antibodies: implications for disease diagnosis and patients' management Sensitivity and specificity enhanced enzyme-linked immunosorbent assay by rational hapten modification and heterogeneous antibody/coating antigen combinations for the detection of melamine in milk, milk powder and feed samples A neutralizing human antibody binds to the N-terminal domain of the Spike protein of SARS-CoV-2 Potent neutralizing antibodies against multiple epitopes on SARS-CoV-2 spike Single-shot Ad26 vaccine protects against SARS-CoV-2 in rhesus macaques COVID-19 vaccines vs variants-determining how much immunity is enough A method comparison of three immunoassays for detection of neutralizing antibodies against SARS-CoV-2 receptor-binding domain in individuals with adenovirus type-5-vectored COVID-19 vaccination