key: cord-0903598-fpltceym
authors: Herroelen, Pauline H; Martens, Geert A; De Smet, Dieter; Swaerts, Koen; Decavele, An-Sofie
title: Humoral Immune Response to SARS-CoV-2: Comparative Clinical Performance of Seven Commercial Serology Tests
date: 2020-08-18
journal: Am J Clin Pathol
DOI: 10.1093/ajcp/aqaa140
sha: daf9ce1cfa1d6726938b1712cb6f005459382657
doc_id: 903598
cord_uid: fpltceym

OBJECTIVES: Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) serology tests are clinically useful to document prior SARS-CoV-2 infections. Data are urgently needed to select assays with optimal sensitivity at acceptable specificity for antibody detection. METHODS: A comparative evaluation was performed of 7 commercial SARS-CoV-2 serology assays on 171 sera from 135 subjects with polymerase chain reaction–confirmed SARS-CoV-2 infection (71 hospitalized patients and 64 paucisymptomatic individuals). Kinetics of IgA/IgM/IgG seroconversion to viral N and S protein epitopes were studied from 0 to 54 days after onset of symptoms. Cross-reactivity was verified on 57 prepandemic samples. RESULTS: Wantai SARS-COV-2 Ab ELISA and Orient Gene COVID-19 IgG/IgM Rapid Test showed superior overall sensitivity for detection of SARS-CoV-2 antibodies. Elecsys Anti-SARS-CoV-2 assay and EUROIMMUN Anti-SARS-CoV-2 combined IgG/IgA showed acceptable sensitivity (>95%) vs the consensus result of all assays from 10 days post onset of symptoms. Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, and Innovita 2019-nCoV Ab rapid test showed least cross-reactivity, resulting in an optimal analytical specificity greater than 98%. CONCLUSIONS: Wantai SARS-COV-2 Ab ELISA and Elecsys Anti-SARS-CoV-2 assays are suitable for sensitive and specific detection of SARS-CoV-2 antibodies from 10 days after onset of symptoms.

intensity of viral replication, sampling quality, and analytical properties of the amplification assay. In addition, insufficient PCR capacity during peak infection rate in overwhelmed health care systems left many patients with milder clinically suspected SARS-CoV-2 infections as well as asymptomatic infections untested.

Serology testing for COVID-19, comprising detection of IgM, IgA, or IgG antibodies to SARS-CoV-2-specific epitopes, might represent an interesting tool to document past SARS-CoV-2 infections, both in individual patients with suspected COVID-19 symptoms or late-stage complications who had no (conclusive) PCR test and at a population level to guide infection control policies. In addition, measuring SARS-CoV-2 antibodies might harbor prognostic value and convey information on protective immunity in vaccination trials.

SARS-CoV-2 shares a 80% overall nucleotide homology with SARS-CoV. 1, 7, 8 In SARS-CoV, the S and N proteins contain the highest density of B-cell epitopes, 9, 10 and in silico analysis indicated that dominant B-cell epitopes share 69% to 100% homology to SARS-CoV-2. It was therefore a logical choice of many commercial developers of SARS-CoV-2 serology kits to target S and N proteins. Antibodies against S protein, composed of a S1 subunit with the receptor-binding domain (RBD) and a S2 subunit that mediates membrane fusion for viral entry, appear additionally interesting because of their proposed correlation with neutralizing antibodies and protective immunity to both SARS-CoV 8 and, based on emerging data, also to SARS-CoV-2. 11, 12 Data on kinetics of humoral immune responses to SARS-CoV-2 are rapidly emerging, but relative sensitivity for detection of antibodies of various commercial assays is questioned. In our study, a cross-platform comparison of 7 commercially available SARS-CoV-2 serology assays was conducted. N and S protein epitopes and different combinations of antibody isotypes in PCR-confirmed COVID-19 patients, with critical and mild disease course at various time points, were targeted. Acceptable performance was defined as minimal sensitivity of 95% for detection of SARS-CoV-2 antibodies vs a consensus estimate and minimal cross-reactivity, defined as analytical specificity of 98%.

Serum samples were obtained from the following cohorts: (1) hospitalized COVID-19 patients-105 serum samples obtained at different time points from 71 patients with PCR-confirmed SARS-CoV-2 infection and admitted for severe COVID-19 pneumonia from March 1 to April 27, 2020, at our tertiary AZ Delta General Hospital in Roeselare, Belgium; and (2) patients with paucisymptomatic SARS-CoV-2 infections-66 serum samples from 64 health care workers with a SARS-CoV-2 infection, PCR-confirmed after developing fever and World Health Organization (WHO)-listed COVID-19 symptoms. These patients were home-quarantined without need for hospitalization. The study was approved by the AZ Delta ethical committee with a waiver of informed consent from the hospitalized COVID-19 patients and with written informed consent from participants with paucisymptomatic SARS-CoV-2 infections.

Cross-reactivity was evaluated on a panel composed of 57 prepandemic serum samples obtained from patients with PCR-confirmed infection by other HCoV respiratory viruses (HCoV 229E, n = 1; HCoV HKU1, n = 3; HCoV OC43, n = 2; HCoV OC43 + adenovirus, n = 1), other pathogens and viruses (n=42), or presence of autoimmune antibodies (n = 8). Serum samples from patients with other HCoV infections ranged from 0 to 39 days after PCR positivity ❚Table 1❚.

All serology assays were used according to manufacturers' protocol using cutoffs specified in package inserts as detailed below.

COVID-19 IgG/IgM Rapid Test (Zhejiang Orient Gene Biotech) is a solid phase immunochromatographic assay for qualitative detection of IgM and IgG antibodies to recombinant N and S proteins. Innovita 2019-nCoV Ab Test (Innovita Biological Technology) is a colloidal gold lateral flow assay for qualitative detection of IgM and IgG antibodies to undisclosed SARS-CoV-2 epitopes. Rapid tests were considered positive if a line was observed for IgM, IgG, or both. Color intensity was not evaluated.

Wantai SARS-COV-2 Ab ELISA (Beijing Wantai Biological Pharmacy Enterprise) is a double-antigen sandwich immunoassay for qualitative detection of all antibody isotypes (IgM, IgA, IgG) against RBD domain of S1 protein. Samples with a cutoff ratio (absorbance of the sample at 450 nm divided by 0.19) higher than 0.9 were considered positive (classifying gray zone results with cutoff ratio 0.9-1.1 as positive). Three indirect ELISAs from EUROIMMUN were tested: Anti-SARS-CoV-2 IgG and IgA assays for semiquantitative detection of IgA and IgG antibodies against S1 protein and All ELISAs were tested using the PhD system (Version EIA 0_16, Bio-Rad Laboratories).

Elecsys Anti-SARS-CoV-2 assay for cobas e601 module (Roche Diagnostics) is a double-antigen sandwich assay for qualitative detection of all antibody isotypes (IgM, IgA, IgG) against N protein (cutoff of 1 cutoff index). LIAISON SARS-CoV-2 S1/S2 IgG (DiaSorin) is an indirect chemiluminescent immunoassay for quantitative detection of IgG antibodies against S1/S2 proteins (cutoff of 12 arbitrary units [AU]/mL, classifying gray zone results of 12-15 AU/mL as positive).

This was done using the Allplex 2019-nCoV assay (Seegene) for E/N/RdRP genes on nasopharyngeal swabs.

Statistical analyses were performed using MedCalc (version 12.2.1). Sensitivities for detection of presence of SARS-CoV-2 antibodies were evaluated on samples obtained from SARS-CoV-2 PCR-positive patients as (1) total fraction of samples showing detectable antibodies and (2) by comparing each individual assay vs consensus outcome obtained by majority of all assays evaluated in this study. χ 2 test was used for comparing proportions for categorical variables. Not normally quantitative variables are expressed as medians (interquartile range [IQR]), and Mann-Whitney test was used to test for statistical differences between various timeframes after onset symptoms. Differences were considered statistically significant if P < .05. Kinetics of seroconversion in individual patients in ❚Figure 1❚ were fitted to a scale from -1 to +1, with 0 representing each assays cutoff by subtracting each assay's cutoff from its raw data signals, and dividing its absolute value by highest (lowest) cutoff-corrected signal for that assay obtained in our data set for positive (negative) samples.

Analytical specificity was evaluated on 57 prepandemic samples from individuals infected with other HCoV viruses (229E/HKU1/OC43), other infectious agents, or with positivity to anti-nuclear factor or rheumatoid factor (Table 1) . Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, EUROIMMUN Anti-SARS-CoV-2 IgG, and Innovita 2019-nCoV Ab Test showed no cross-reactivity ❚Table 2❚. EUROIMMUN Anti-SARS-CoV-2 IgA and Orient Gene COVID-19 IgG/IgM Rapid Test showed cross reactivity with common cold HCoV viruses, resulting in respective analytical specificities of 91.1% and 92.9%. LIAISON SARS-CoV-2 S1/S2 IgG (96.4% analytical specificity) was the only to show interference by rheumatoid factor (Table 1) .

Sensitivities for detection of SARS-CoV-2 antibodies were compared on 171 samples obtained from 135 subjects, all with PCR-confirmed SARS-CoV-2 infections, pooled or grouped in two distinct cohorts: hospitalized and paucisymptomatic COVID-19 patients. Hospitalized patients included 105 samples from 71 patients hospitalized for severe COVID-19 disease, all with very high level of suspicion of COVID-19 pneumonia on chest computed tomography (COVID-19 Reporting and Data System [CO-RADS] score = 5) 13 : 48 males (median age, 65 years; IQR, 53-80) and 23 females (median age, 79 years; IQR, 67-86). Serum samples ranged from 0 to 39 days after patient-reported symptom onset. Paucisymptomatic patients included 66 samples from 64 health care workers with mild (n = 61) or no (n = 3) WHO-listed COVID-19 symptoms: myalgia (present in 62.5%), fever (60.9%), dry cough (56.2%), dyspnea (40.6%), severe fatigue (35.9%), headaches (30.0%), loss of smell or taste (26.6%), or diarrhea (18.8%). None of these patients were hospitalized. Serum samples ranged from 11 to 54 days after patientreported symptom onset.

Sensitivity was calculated for different patient groups (all patients, hospitalized and paucisymptomatic patients). First, vs SARS-CoV-2 PCR (100% of samples from PCR+ patients) as reference, by measuring the percentage of samples showing antibody titers above the respective assay's cutoff (Table 2) . Second, by comparing each individual assay to the consensus outcome of the majority of 7 tested assays ❚Table 3❚. Wantai SARS-COV-2 Ab ELISA showed highest overall sensitivity for detection of SARS-CoV-2 antibodies: 86.4% (95% confidence interval [CI], 80.3%-91.2%) vs PCR and 100% (95% CI, 97.3%-100%) vs consensus at all time points in both patient cohorts. Its sensitivity was significantly higher (P < .05) than all other assays with the exception of Orient Gene COVID-19 IgG/IgM Rapid Test and EUROIMMUN Anti-SARS-CoV-2 IgG and IgA combined. In a real-world clinical setting, serology assays might be used at later time stages, eg, more than 20 days after onset of symptoms or to document past SARS-CoV-2 infection in paucisymptomatic patients. In these patients, 4 assays showed clinically acceptable sensitivity for detection of SARS-CoV-2 antibodies above 95% vs consensus result (Table 3) : Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, EUROIMMUN Anti-SARS-CoV-2 IgG and IgA combined, and Orient Gene COVID-19 IgG/IgM Rapid Test. In comparison with all other assays, LIAISON SARS-CoV-2 S1/S2 IgG showed significantly (P < .05) lower sensitivities of 83.6% (95% CI, 72.5%-91.5%) vs consensus (Table 3) 

We compared timing of detection of antibodies of the ELISA/CLIA assays on consecutive blood samples of 8 critically ill patients admitted to intensive care units (Figure 1) . In all 8 patients, Wantai SARS-COV-2 Ab ELISA was first to exceed the predefined assay cutoff, followed by the EUROIMMUN Anti-SARS-CoV-2 IgA assay. In this cohort of intensive care patients, of the N-targeting assays, EUROIMMUN Anti-SARS-CoV-2-NCP(IgG) provided positive results more rapidly than Elecsys Anti-SARS-CoV-2 assay. LIAISON SARS-CoV-2 S1/S2 IgG was last to detect seroconversion. Seroconversion rates were additionally studied by a pooled analysis in samples from different patients, grouped according to the timeframe after symptom onset ranging from less than 10 days, 10 to 20 days, or more than 20 days post onset of symptoms (Tables 2 and 3 ). All tests except Wantai SARS-COV-2 Ab ELISA showed a significantly higher positivity rate between 10 and 20 days post onset of symptoms as compared to less than 10 days post onset of symptoms (P < .05). No significant differences were observed in positivity rates between 10 and 20 days post onset of symptoms and more than 20 days post onset of symptoms ( Table 2 ), indicating that serology testing can be performed starting from 10 days after onset symptoms. In samples less than 10 days after onset of symptoms, all from hospitalized patients, the Wantai SARS-COV-2 Ab ELISA outperformed all other assays, with a sensitivity of 100% (95% CI, 88.1%-100%) vs consensus and 75.5% (95% CI, 61.7%-86.2%) vs PCR, which was significantly lower than its performance in samples from patients greater than 20 days post onset of symptoms (P < .05).

For the assays with acceptable overall sensitivity above 95% (Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, EUROIMMUN Anti-SARS-CoV-2 IgG and IgA combined, and Orient Gene COVID-19 IgG/IgM Rapid Test) a good overall concordance was seen in samples from patients greater than 10 days post onset of symptoms, with 87.7% and 3.5% of samples positive or negative respectively with all 4 methods. No clear differences were observed in kinetics of appearance of antibodies to S or N epitopes. Beyond 10 days, only 1.4% (1/71) of hospitalized and 4.7% (3/64) paucisymptomatic patients developed no detectable antibodies.

In this study we report on the clinical performance characteristics of 7 commercially available serology tests for detection of antibodies against SARS-CoV-2 S protein (S-RBD total antibodies, S1/S2 IgG, S1 IgA and IgG), N protein (N total antibodies, N IgG), and both proteins (N/S IgM and IgG). To our knowledge, this study is the first to report performance of Elecsys Anti-SARS-CoV-2 assay on the cobas e601 module. We specifically investigated their relative value as a complementary tool to screen for prior SARS-CoV-2 infection in individuals that were not (conclusively) tested by PCR in early stage of active viral replication up to 10 days after onset of symptoms. As a working definition for acceptable performance, we propose that such an assay should combine a minimal sensitivity for detection of SARS-CoV-2 antibodies of 95% vs a consensus estimate and a high analytical specificity above 98% in samples taken 20 days or more after onset of symptoms, also in subjects who experienced mild SARS-CoV-2 symptoms. Based on these criteria, Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, and Innovita 2019-nCoV Ab Test all showed acceptable analytical specificity. In terms of sensitivity for detection of SARS-CoV-2 antibodies vs consensus result obtained by all tests, Wantai SARS-COV-2 Ab ELISA, Elecsys Anti-SARS-CoV-2 assay, EUROIMMUN Anti-SARS-CoV-2 IgG combined with IgA, and Orient Gene COVID-19 IgG/IgM Rapid Test were acceptable. Overall, only Wantai SARS-COV-2 Ab ELISA and Elecsys Anti-SARS-CoV-2 assay fulfilled the proposed acceptance criteria, with Wantai SARS-COV-2 Ab ELISA clearly outperforming all other evaluated assays.

A strength of this study is that the parallel evaluation of several kits allowed a reliable direct comparison of clinical performance using cutoffs provided by manufacturers. Also, our patient cohorts, including not only severe COVID-19 patients but also a sizeable cohort of mild SARS-CoV-2 infections, provide a good estimate on assays' performances in the intended target population. We observed no notable differences in timing of seroconversion between severe and milder SARS-CoV-2 infections.

There are limitations to our study. Cross-reactivity analysis might require more extensive exploration.

A higher number of sera from patients with PCRconfirmed HCoV infections and other common cold viruses need to be investigated. Our study did not include a sizeable cohort of fully asymptomatic SARS-CoV-2 infections, and it focused solely on qualitative analysis. Therefore, no investigation of differences in assays' performance for quantification of antibody titers was performed.

In critically ill COVID-19 patients, SARS-CoV-2 antibody levels were reported to correlate to disease severity 1 by triggering bradykinin and complement activation pathways. The assays evaluated here show large variations in their dynamic range, ranging from a good linearity for LIAISON SARS-CoV-2 S1/S2 IgG 14 to a limited dynamic range with rapid signal saturation for Wantai SARS-COV-2 Ab ELISA. With a sample volume input of 100 μL that is 10 to 20 times higher than the other evaluated assays, Wantai SARS-COV-2 Ab ELISA is clearly designed toward high sensitivity for detection of SARS-CoV-2 antibodies by maximal antibody capture. Caution is needed when comparing (semi)quantitative estimates of antibody titers across platforms before certified standards with known titers become available.

Our data are compatible with other cross-platform evaluations 15 indicating superior performance of Wantai SARS-COV-2 Ab ELISA as compared to EUROIMMUN Anti-SARS-CoV-2 IgG and IgA. The results are, however, discrepant with another study reporting a sensitivity of 100% and 99% specificity for LIAISON SARS-CoV-2 S1/ S2 IgG 14 obtained on a small set of 125 samples including only 40 PCR-confirmed patients and after receiver operating characteristic optimization of assay cutoffs. Since we observed considerable lot-to-lot variations in raw signals of the 2 LIAISON SARS-CoV-2 S1/S2 IgG kits tested, we believe that caution is warranted and cutoffs should only be optimized on better data sets and proper assessment of different lots.

It was reported that antibodies against S protein appear later in infection than antibodies against N protein. 1, 9 We also observed faster seroconversion of N vs S1 targeting IgG in EUROIMMUN assays. On the other hand, we observed a much faster seroconversion of total antibodies (IgA/IgM/IgG) against S-RBD (Wantai) than N protein (Elecsys). Within the same epitope/assay format (EUROIMMUN to S1), IgA antibodies clearly precede IgG. Overall, our data suggest that timing of seroconversion depends more on assay design, recombinant viral epitope, and antibody isotypes covered, and that overall sensitivity for detection of antibodies is likely enhanced when both IgA and IgG isotypes are measured.

In conclusion, this study supports clinical use of both Wantai SARS-COV-2 Ab ELISA and Elecsys Anti-SARS-CoV-2 assay for sensitive and specific screening of SARS-CoV-2 antibodies from 10 days after onset of symptoms.

Corresponding author: Geert Martens, MD, PhD; geert.martens@ azdelta.be.

Anti-SARS-CoV-2-NCP(IgG) and Innovita

SARS-CoV-2 serological analysis of COVID-19 hospitalized patients, paucisymptomatic individuals and blood donors

Vitamin D deficiency as risk factor for severe COVID-19 : a convergence of two pandemics

Antibody responses to individual proteins of SARS coronavirus and their neutralization activities

Molecular, serological, and biochemical diagnosis and monitoring of COVID-19: IFCC taskforce evaluation of the latest evidence

Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases

Diagnostic power of chest CT for COVID-19: to screen or not to screen

Genomic characterization of the 2019 novel human-pathogenic coronavirus isolated from a patient with atypical pneumonia after visiting Wuhan

Beyond the spike: identification of viral targets of the antibody responses to SARS-CoV-2 in COVID-19 patients

Serological assays for emerging coronaviruses: challenges and pitfalls

A sequence momology and bioinformatic approach can predict candidate targets for immune responses to SARS-CoV-2

Towards the next phase: evaluation of serological assays for diagnostics and exposure assessment

Severe acute respiratory syndrome coronavirus 2−specific antibody responses in coronavirus disease 2019 patients

CO-RADS-a categorical CT assessment scheme for patients with suspected COVID-19: definition and evaluation

Validation of a chemiluminescent assay for specific SARS-CoV-2 antibody

Evaluation of nine commercial SARS-CoV-2 immunoassays