key: cord-0937953-m6b97t14
authors: Jung, Kiwook; Shin, Sue; Nam, Minjeong; Hong, Yun Ji; Roh, Eun Youn; Park, Kyoung Un; Song, Eun Young
title: Performance evaluation of three automated quantitative immunoassays and their correlation with a surrogate virus neutralization test in coronavirus disease 19 patients and pre‐pandemic controls
date: 2021-08-08
journal: J Clin Lab Anal
DOI: 10.1002/jcla.23921
sha: 7a9a790edc847a8d9e9928e97498c449b7f3e4f4
doc_id: 937953
cord_uid: m6b97t14

BACKGROUND: SARS‐CoV‐2 pandemic is currently ongoing, meanwhile vaccinations are rapidly underway in some countries. The quantitative immunoassays detecting antibodies against spike antigen of SARS‐CoV‐2 have been developed based on the findings that they have a better correlation with the neutralizing antibody. METHODS: The performances of the Abbott Architect SARS‐CoV‐2 IgG II Quant, DiaSorin LIAISON SARS‐CoV‐2 TrimericS IgG, and Roche Elecsys anti‐SARS‐CoV‐2 S were evaluated on 173 sera from 126 SARS‐CoV‐2 patients and 151 pre‐pandemic sera. Their correlations with GenScript cPass SARS‐CoV‐2 Neutralization Antibody Detection Kit were also analyzed on 173 sera from 126 SARS‐CoV‐2 patients. RESULTS: Architect SARS‐CoV‐2 IgG II Quant and Elecsys anti‐SARS‐CoV‐2 S showed the highest overall sensitivity (96.0%), followed by LIAISON SARS‐CoV‐2 TrimericS IgG (93.6%). The specificities of Elecsys anti‐SARS‐CoV‐2 S and LIAISON SARS‐CoV‐2 TrimericS IgG were 100.0%, followed by Architect SARS‐CoV‐2 IgG II Quant (99.3%). Regarding the correlation with cPass neutralization antibody assay, LIAISON SARS‐CoV‐2 TrimericS IgG showed the best correlation (Spearman rho = 0.88), followed by Architect SARS‐CoV‐2 IgG II Quant and Elecsys anti‐SARS‐CoV‐2 S (all rho = 0.87). CONCLUSIONS: The three automated quantitative immunoassays showed good diagnostic performance and strong correlations with neutralization antibodies. These assays will be useful in diagnostic assistance, evaluating the response to vaccination, and the assessment of herd immunity in the future.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), first reported in Wuhan, China in 2019, caused a worldwide outbreak that is currently ongoing. 1 Coronavirus disease , the infectious disease caused by SARS-CoV-2, became not only an unprecedented threat to public health worldwide but also a tragic shock to the economy across the globe. 2 The absence of specific treatment options proved to be effective against SARS-CoV-2 aggravated the affair. 3 This has resulted in the heightened importance of SARS-CoV-2 diagnostic testing as quarantine and social distancing have become the primary strategies for control of COVID- 19. 4 Molecular testing, especially RT-PCR, a reliable tool detecting the active SARS-CoV-2 infection, is the first option for the COVID-19 diagnosis. 5, 6 And serologic testing, a secondary weapon in the diagnostic arsenal for COVID-19, was used as complementary to the RT-PCR in the area where RT-PCR has its limitations. Because serologic testing for COVID-19 has its advantages such as cost-effectiveness, short turnaround time, high-throughput, ability to detect past infection, and usefulness in resource-limited areas. 7 Recently, the countermeasure strategy against COVID-19 has stepped up from detection and quarantine of infection to active achievement of herd immunity through vaccination, as several vaccines have been approved for emergency use by the government in Europe and the United States and vaccinations are rapidly underway in some countries. [8] [9] [10] In line with these shifts, the importance of tests detecting antibodies for SARS-CoV-2, especially neutralizing antibodies representing the protective ability of host immunity, is further emphasized.

The majority of SARS-CoV-2 serologic assays used whole or some parts of spike protein (S1 subunit, S2 subunit, and receptor binding domain located in S1 subunit) or nucleocapsid (N) protein as target antigens. 11 Previous studies reported that assays targeting spike protein showed a better correlation with virus neutralization assay compared to nucleocapsid protein. 12, 13 Recently, commercial manufacturers launched the quantitative SARS-CoV-2 antibody assays using spike protein as a target antigen, which can be a pivotal tool assessing the effect of vaccination. Abbott Architect anti-SARS-CoV-2 IgG II Quant, new quantitative SARS-CoV-2 IgG immunoassay released by Abbott, received CM mark approval from the EU government. Roche also released their new quantitative assay, Elecsys anti-SARS-CoV-2 S, targeting receptor binding domain (RBD) of S1 subunit. And DiaSorin LIAISON SARS-CoV-2 TrimericS IgG has been developed based on the observation that trimer form of spike protein showed greater sensitivity for the detection of SARS-CoV-2 antibodies. 14 Many studies are conducted regarding the clinical performances of the previous version of anti-SARS-CoV-2 assays against N protein (Abbott or Roche) and against S1/S2 subunit (Diasorin). [15] [16] [17] [18] However, clinical performances of newly launched SARS-CoV-2 antibody assays against S1 RBD (Abbott or Roche) or TrimericS (Diasorin) have not been evaluated thoroughly. To the best of our knowledge, the performance of Abbott Architect anti-SARS-CoV-2 IgG II Quant has not been fully evaluated so far. The performances of Elecsys anti-SARS-CoV-2 S have been reported in comparison with the previous version (Elecsys anti-SARS-CoV-2 against N antigen), showed better sensitivity than Elecsys anti-SARS-CoV-2 against N. 19, 20 The performance of LIAISON SARS-CoV-2 TrimericS IgG has been once reported 21 and showed better performance than previous version of LIAISON SARS-CoV-2 against S1/S2. [15] [16] [17] However, the superiority of clinical performance can be evaluated precisely when performed in the same population. We assessed the clinical performance of three newly developed anti-SARS-CoV-2 assays in the same subjects.

Meanwhile, virus neutralization assay using live SARS-CoV-2 is the gold standard method for assessing neutralizing antibodies. But its utility is limited because it is labor-intensive, time-consuming, and requires specialized facilities such as biosafety level 3. 4 For this reason, researchers tried to develop alternatives that are more appropriate for large-scale use in clinical laboratories. 22

Kit is an enzyme-linked immunosorbent assay (ELISA) based surrogate virus neutralization test (sVNT) that mimics the reaction of human ACE2 receptor and RBD. It has been reported that cPass SARS-CoV-2 Neutralization Antibody test presented an excellent correlation with cell-culture-based virus neutralization assays and could be a useful measure of virus-neutralizing activity. 23, 24 The correlations of cPass SARS-CoV-2 Neutralization Antibody test with three automated anti-SARS-CoV-2 assays (Mindray CL-900i against S and N, BioMerieux VIDAS 3 against RBD, and Diasorin LIAISON SARS-CoV-2 against S1/S2) have been reported with the best correlation in VIDAS 3 (r = 0.75), followed by LIAISON S1/ S2 (r = 0.66) and Mindray CL-900i (r = 0.57) 25 . However, the cor- We evaluated their clinical performance and quantitative correlation with cPass SARS-CoV-2 Neutralization Antibody test. pre-pandemic sera were tested. Out of the 151 sera, 98 were from 

Three fully automated commercial immunoassays were evaluated.

The specifications of the three immunoassays are summarized in Signal inhibition was calculated as follow:

The test results were interpreted as positive when the percent signal inhibition was ≥30%, which is the cut-off for signal inhibition claimed by the manufacturer.

The precision assessment was performed on three quantitative assays, according to CLSI EP15-A3 protocol, 27 using one quality control material and two pooled patient sera for five consecutive days, five times a day. Repeatability and within-laboratory precision were estimated using ANOVA and compared to values claimed by the manufacturers.

Linearity assessment was performed on three quantitative assays, according to CLSI EP6-A protocol. 28 Two patient sera with high (H) and low (L) concentration were mixed at ratios of 4H, 1L + 3H, 2L + 2H, 3L + 1H, and 4L. All levels are measured in duplicates.

For three immunoassays, sensitivity and specificity were calculated.

The sensitivity of the subgroup sampled 14 days after the onset of symptoms was also calculated and compared with the manufacturer's claim. It is in line with the recommendation from infectious diseases society of America guidelines on the Diagnosis of COVID-19 that suggests against using serologic testing to diagnose SARS-CoV-2 infection during the first two weeks (14 days) following symptom onset. 29 The concordances between the three immunoassays and cPass were assessed using overall, positive and negative percent agreement as well as Cohen's kappa statistics. Cohen's kappa is a robust statistic of inter-rater reliability, useful for assessing the level of agreement between two diagnostic assays. Ranging between 0 and 1, a kappa value <0.40 represents poor agreement, 0.40-0.59 represents fair agreement, 0.60-0.74 represents good agreement, and ≥0.75 represents excellent agreement. 30 We evaluated the correlations of the quantitative value of three immunoassays with each other and with % inhibition value of cPass using Spearman's rank-order correlation coefficient (rho). All statistical analyses were performed by using R version 4.0.5 (R foundation for statistical Computing).

The clinical performances of three immunoassays are shown in 

The repeatability and within-laboratory imprecision for three immunoassays are shown in Table 3 . The within-laboratory precisions of Abbott Quant and Roche S were all <4.0%. The within-laboratory precisions of DiaSorin TrimericS were 2.9-8.2%, which were slightly larger than that claimed by the manufacturer.

The linearity assessment of three immunoassays ( Figure S1 ) 

The correlations between results from three immunoassays were shown in Figure 1 

The concordances between the qualitative results of three immunoassays and cPass SARS-CoV-2 neutralization test in SARS-CoV-2 positive patients are shown in Table 4 . The positive percent 

Architect (Figure 1 ).

Receiver operating characteristics (ROC) curve analysis was performed on three immunoassays (Figure 2 

In this study, we compared three commercially available automated were 70.0%~85.3%, slightly lower than those of Abbott or Roche anti-SARS-CoV-2 (N). [15] [16] [17] In the report evaluating two Roche anti-SARS-CoV-2 assays against N or S simultaneously, 19 anti-SARS-CoV-2 S showed higher sensitivity than anti-SARS-CoV-2 (against N) (93.0% vs. 89.0%). In a recent report, the sensitivity of DiaSorin anti-SARS-CoV-2 Trimeric S was 99.4%, which was higher than the previous version of DiaSorin anti-SARS-CoV-2 against S1/S2. 21 In our study, the sensitivity was highest in Roche S and Abbott Quant (96.0%), followed by DiaSorin Trimeric S (93.6%). All three immunoassays showed higher sensitivities than prior reports of the previous The sensitivity of cPass NT has been reported higher (93%) than those of Abbott anti-SARS-CoV-2 (N) (89%) or Roche anti-SARS-CoV-2 total (N) (83%). 17 In our study, cPass showed similar sensitivity (94.8%) with the other three immunoassays (93.6%~96.0%), which is consistent with the previous reports, considering the lower sensitivity of Abbott or Roche anti-SARS-CoV-2 (N) assays in prior studies. [15] [16] [17] [18] In the specificity test conducted with 151 pre-pandemic samples, all three immunoassays showed remarkable specificity with only one positive result from Abbott Quant. These results were consistent with manufacturer's claim and previous studies, which report superior sensitivity in the new version of anti-SARS-CoV-2 assays (range 99.8%-100.0%). [19] [20] [21] In the previous version of three immunoassays, DiaSorin anti-SARS-CoV-2 against S1/S2 showed slightly lower specificities than Abbott anti-SARS-CoV-2 (N) or Roche anti-SARS-CoV-2 total (N). [15] [16] [17] In the new DiaSorin anti-SARS-CoV-2 against trimeric S, specificity was excellent (99.8%). 21 The sensitivities of both the previous and new version of Roche anti-SARS-CoV-2

were excellent (100.0% for both). 19, 20 Imprecision of the new Roche anti-SARS-CoV-2 against S has been reported as 1.06% at 9.06 U/ml. 34 

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Kiwook Jung https://orcid.org/0000-0001-8411-7473

Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia

Coronavirus outbreak and its impacts on global economy: the role of social network sites

Towards a more effective strategy for COVID-19 prevention (Review)

A high-throughput neutralizing antibody assay for COVID-19 diagnosis and vaccine evaluation

Detection technologies and recent developments in the diagnosis of COVID-19 infection

Diagnostic approaches in COVID-19: clinical updates

Update on serologic testing in COVID-19

A new vaccine to battle covid-19

Mass-vaccination sites -An essential innovation to curb the covid-19 pandemic

The European medicines agency's EU conditional marketing authorisations for COVID-19 vaccines

Review of current advances in serologic testing for COVID-19

Longitudinal serological analysis and neutralizing antibody levels in coronavirus disease 2019 convalescent patients

Quantification of SARS-CoV-2 antibodies with eight commercially available immunoassays

Changes in SARS-CoV-2 spike versus nucleoprotein antibody responses impact the estimates of infections in population-based seroprevalence studies

Evaluation of four laboratorybased SARS-CoV-2 IgG antibody immunoassays

Comparison of 16 serological SARS-CoV-2 immunoassays in 16 clinical laboratories

A new SARS-CoV-2 dualpurpose serology test: highly accurate infection tracing and neutralizing antibody response detection

Evaluation of four commercial, fully automated SARS-CoV-2 antibody tests suggests a revision of the Siemens SARS-CoV-2 IgG assay

Longitudinal assessment of SARS-CoV-2 Antinucleocapsid and antispike-1-RBD antibody testing following PCR-detected SARS-CoV-2 infection

Head-to-head comparison of two rapid high-throughput automated electrochemiluminescence immunoassays targeting total antibodies to the SARS-CoV-2 nucleoprotein and spike protein receptor binding domain

Evaluation of the automated LIAISON ® SARS-CoV-2 TrimericS IgG assay for the detection of circulating antibodies

A SARS-CoV-2 surrogate virus neutralization test based on antibody-mediated blockage of ACE2-spike protein-protein interaction

Validation and clinical evaluation of a SARS-CoV-2 surrogate virus neutralisation test (sVNT)

Validation of a commercially available indirect assay for SARS-CoV-2 neutralising antibodies using a pseudotyped virus assay

Diagnostic efficiency of three fully automated serology assays and their correlation with a novel surrogate virus neutralization test in symptomatic and asymptomatic SARS-COV-2 individuals

Clinical Management of COVID-19: Interim Guidance

User Verification of Precision and Estimation of Bias; Approved Guideline

CLSI. Evaluation of Linearity of Quantitative Measurement Procedures

Infectious Diseases Society of America guidelines on the diagnosis of COVID-19: serologic testing

Interrater reliability: the kappa statistic

Antibody responses to SARS-CoV-2 in patients with novel coronavirus disease 2019

Differences in antibody kinetics and functionality between severe and mild severe acute respiratory syndrome Coronavirus 2 infections

Lack of antibodies to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in a large cohort of previously infected persons

Quantitative measurement of anti-SARS-CoV-2 antibodies: analytical and clinical evaluation

Evaluation of a SARS-CoV-2 surrogate virus neutralization test for detection of antibody in human, canine, cat, and hamster sera