key: cord-1016266-ugvwjnyo
authors: Favresse, Julien; Cadrobbi, Julie; Eucher, Christine; Elsen, Marc; Laffineur, Kim; Dogné, Jean‐Michel; Douxfils, Jonathan
title: Clinical performance of three fully automated anti‐SARS‐CoV‐2 immunoassays targeting the nucleocapsid or spike proteins
date: 2020-12-01
journal: J Med Virol
DOI: 10.1002/jmv.26669
sha: 63ad41724be3eaee89e6d245a3b7a394810e3914
doc_id: 1016266
cord_uid: ugvwjnyo

This study assesses the clinical performance of three anti‐SARS‐CoV‐2 assays, namely EUROIMMUN anti‐SARS‐CoV‐2 nucleocapsid (IgG) ELISA, Elecsys anti‐SARS‐CoV‐2 nucleocapsid (total antibodies) assay, and LIAISON anti‐SARS‐CoV‐2 spike proteins S1 and S2 (IgG) assay. One hundred and thirty‐seven coronavirus disease 2019 (COVID‐19) samples from 96 reverse‐transcription polymerase chain reaction confirmed patients were chosen to perform the sensitivity analysis. Non‐SARS‐CoV‐2 sera (n = 141) with a potential cross‐reaction to SARS‐CoV‐2 immunoassays were included in the specificity analysis. None of these tests demonstrated a sufficiently high clinical sensitivity to diagnose acute infection. Fourteen days since symptom onset, we did not find any significant difference between the three techniques in terms of sensitivities. However, Elecsys performed better in terms of specificity. All three anti‐SARS‐CoV‐2 assays had equivalent sensitivities 14 days from symptom onset to diagnose past‐COVID‐19 infection. We also confirmed that anti‐SARS‐CoV‐2 determination before Day 14 is of less clinical interest.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of coronavirus disease 2019 (COVID- 19) , has led to significant morbidity and mortality. 1 The number of confirmed cases exceeds 7.8 million and the number of deaths worldwide stands at 431,541. 2 The considered standard method of reference for the diagnosis of SARS-CoV-2 infection is (real-time) reverse-transcription polymerase chain reaction (RT-PCR) in respiratory samples. 3 However, the accuracy of the method depends on several factors including pre-analytical variables like sample type, collection, transport, and storage. 4 The time since infection and the viral load are other factors affecting the sensitivity of the RT-PCR. 3 In addition, RT-PCR is not able to detect past infection 5 and the throughput of RT-PCR is also limited because it requires a high workload, skillful operators, expensive instrumentation, and crucial biosafety measures. 6 Access to RT-PCR tests remains limited in many countries worldwide while the virus is present in 188 countries. 7 The detection of anti-SARS-CoV-2 antibodies represents an additional method for the diagnosis of COVID-19, especially in patients who present late, with a low viral load. 8 Detection of anti-SARS-CoV-2 antibodies is also useful to identify convalescent plasma donors and to screen the population to determine seroprevalence. 9, 10 A wide range of serology immunoassays has therefore been developed to complement the RT-PCR, with different SARS-CoV-2 antigen targets and formats. 3 More than 100 manufacturers have notified that they are offering or plan to offer serological testing. 11 Due to the widespread dissemination of these methods and the limited experience with these new assays, it is essential for laboratories to independently validate these methods to assure they are in line with the expected analytical and clinical performance. 3, 5, 8, [12] [13] [14] [15] This is also the reason why some national authorities are planning broad validation campaigns to ensure they will offer the population approved and controlled immunoassays which are the cornerstone to fight this pandemic.

The aim of the present study is to assess and compare the clinical performance of three fully automated anti-SARS-CoV-2 immunoassays, namely: EUROIMMUN anti-SARS-CoV-2-nucleocapsid (IgG) ELISA, Elecsys anti-SARS-CoV-2-nucleocapsid (total antibodies) assay, and LIAISON anti-SARS-CoV-2 spike proteins S1 and S2 (IgG) assay. The study population displayed the following characteristics:

There were 45 females and 51 males aged 24 to 93 years (mean age = 63 years). Information on the days since the onset of symptoms was retrieved from medical records. Symptoms included fever, cough, fatigue, muscle aches, chest pain or pressure, difficulty breathing or shortness of breath, headache, sore throat, diarrhea, loss of taste, and loss of smell. Fever was the most frequent symptom (68.1%), followed by cough (60.4%), fatigue (58.2%), difficulty breathing (45.1%), and muscle aches (31.9%).

Non-SARS-CoV-2 sera with a potential cross-reaction to the SARS-CoV-2 immunoassay were collected before December 2019.

Thirty-seven samples were kindly provided by the Department of Laboratory Medicine of Iris Hospitals South in Brussels. Samples were stored in the laboratory serum biobank at −20°C. Frozen samples were thawed one hour at room temperature on the day of the analysis. Re-thawed samples were vortexed before the analysis.

Three anti-SARS-CoV-2 immunoassays were evaluated.

The anti-SARS-CoV-2-nucleocapsid ELISA (EUROIMMUN Medizinische Labordiagnostika AG) is used for the in vitro semiquantitative detection of IgG (also IgA and IgM, according to the insert kit of the manufacturer) to SARS-CoV-2 in human serum and plasma. 19 All measurements were performed on the EUROIMMUN Analyzer I-2P®. The result of a sample is given in the form of a ratio (extinction of patient sample/extinction of calibrator). According to the manufacturer, a ratio < 0.80 is considered negative, a ratio ≥ 0.80 to < 1.10 considered borderline, and a ratio ≥ 1.10 considered positive. 19 The Elecsys anti-SARS-CoV-2 nucleocapsid electrochemiluminescent immunoassay (ECLIA) (Roche Diagnostics) is used for the in vitro qualitative detection of total antibodies (including IgG) to SARS-CoV-2 in human serum and plasma. All measurements were performed on the Cobas® e801 module. The test result is given as a cut-off index (COI). According to the manufacturer, a result <1.00 is considered negative while a result ≥1.00 is considered positive. 5 The LIAISON SARS-CoV-2 spike proteins S1/S2 assay (DiaSorin) is used for the in vitro quantitative detection of IgG to SARS-CoV-2 in human serum and plasma. All measurements were performed on the LIAISON-XL analyzer. The test result is given as arbitrary units per ml (AU/ml). According to the manufacturer, a result < 12.0 is considered negative, a result ≥12.0 to <15.0 considered borderline, and a result ≥5.0 considered positive. 15 Only one calibration curve was done, and one batch of reagent was used for each of these platforms. 

The diluent provided by the manufacturer (diluent universal) was used as a blank sample to determine the limit of blank (LOB), detection (LOD), and quantification (LOQ). The LOB has been determined by running the blank sample on three separate occasions to verify that the results are well < 0.80. The LOD and LOQ have been determined by running 30 analyses of the blank sample using the following equations according to the SH GTA 04 document-revision 1 of the COFRAC. 21 -LOD = mean of the 30 measurements + 3 × standard deviation -LOQ = mean of the 30 measurements + 10 × standard deviation.

Linearity was evaluated according to CLSI EP-06. A sample with high total antibody levels (i.e., 9.52) was analyzed and diluted by a factor of 2 on 5 consecutive dilutions. The manufacturer's diluent was used for the dilution. Observed values were compared to the expected ones and polynomial regression was calculated.

A sample with a high IgG value (i.e., 8.92) was run in triplicate (A1, A2, A3) and followed by a negative sample (i.e., 0.10) also run in triplicate (B1, B2, B3). The carry-over formula used is (B1-B3)/ (A3-B3) × 100. A carry-over below 1% is considered negligible.

One-hundred forty-one non-SARS-CoV-2 sera were analyzed for determining the cross-reactivity and establishing specificity.

Thirty-eight sera from COVID-19 negative healthy subjects and 103 sera from patients with a potential cross-reaction to the SARS-CoV-2 immunoassay were included in this study. Potential Toxoplasma gondii (n = 11), influenza A antibodies (n = 4), influenza A and B (n = 1), high level of total IgG (17.40 g/L) (normal range, 7.00-16.00 g/L) (n = 1), both high levels of total IgM (5.26 g/L; normal range, 0.4-2.3 g/L) and total IgG (28.67 g/L) (n = 1), rheumatoid factor (n = 6), urinary tract infection with Escherichia coli (n = 1), urinary tract infection with Klebsiella oxytoca (n = 1). All these samples were collected before the COVID-19 pandemic and were stored at −20°C.

The calculation of the specificity was stratified by excluding these cross-reactive samples from the pool of healthy subjects and by combining the two cohorts. 

Descriptive statistics were used to analyze the data. Sensitivity was 

The calculated specificity was 100% (141 of 141) (95% CI, 97.4%-100%) by using the manufacturer's cut-off (i.e., ≥1.00). Using an optimized cut-off (i.e., >0.165 COI) did not alter the specificity. The calculated specificity was 100% (95% CI, 90.8%-100%) and 100% (95% CI, 96.5%-100%) for healthy volunteers and cross-reactive samples, respectively, by using the manufacturer's cut-off (Table 1) . 95.0%-99.8%). Using optimized cut-off (ratio > 3.94 AU/ml), specificity was 91.5% (129 of 141) (95% CI, 85.6%-95.5%). The calculated specificity was 100% (95% CI, 90.8%-100%) and 97.1% (95% CI, 91.7%-99.4%) for healthy volunteers and cross-reactive samples, respectively, by using the manufacturer's cut-off (Table 1) .

The calculated sensitivities classified according to different time categories since symptom onset are represented in Table 1 .

Fourteen days before symptom onset, the sensitivities (ranging from 70.4% to 85.2%) were not high enough to be reliably used in clinical practice, especially considering the LIAISON IgG assay.

After 2 weeks since symptom onset, the sensitivity was 90.8% (79 of 87) (95% CI, 82.7%-96.0%) by using the cut-off provided by the manufacturer and considering borderline results (n = 1) as positive.

Using the optimized cut-off, the sensitivity (i.e., ratio > 0.40) was 95.4% (83 of 87) (95% CI, 86.6%-98.7%) (Figure 1 ). The sensitivity increased to 97.5% (39 of 40) (95% CI, 86.8%-99.9%) from 28 days since symptom onset (Table 1 ).

After 2 weeks since symptom onset, the sensitivity was 92.0% (80 of 87) (95% CI, 84.1%-96.7%) by using the cut-off provided by the manufacturer. Using the optimized cut-off, the sensitivity (i.e., >0.165 COI) was 95.4% (83 of 87) (95% CI, 88.6%-98.7%) (Figure 1 ). The sensitivity increased to 100% (40 of 40) (95% CI, 91.2%-100%) from 28 days since symptom onset (Table 1 ).

After 2 weeks since symptom onset, the sensitivity was 88.5% (77 of 87) (95% CI, 79.9%-94.4%) by using the cut-off provided by the manufacturer and considering borderline results (n = 1) as positive.

Using the optimized cut-off, the sensitivity (i.e., >3.94 AU/ml) was 96.6% (84 of 87) (95% CI, 90.3%-99.3%) (Figure 1) The EUROIMMUN IgG assay is also sensitive to IgA and IgM, according to the insert kit of the manufacturer. programs, and COVID-19 vaccine development. 5, 6, 8, 14, 22 To date, peerreviewed data concerning the performance of SARS-CoV-2 immunoassays remains limited, but it is crucial for society to be confident in the results of these assays. Therefore, independent validations of these methods before broad introduction into routine clinical practice is mandatory, given the limited experience of the scientific community with these new assays. 6, 10, 14, [23] [24] [25] [26] [27] We report here the external validation of the EUROIMMUN anti-SARS-CoV-2-nucleocapsid (IgG) ELISA. Our results show satisfactory analytical performance. Repeatability and reproducibility studies determined on two different pools of sera from patients and two internal quality controls were ≤7.3% and ≤7.6%, respectively. The carry-over was negligible, and we found a LOQ of 0.164, which is lower than the optimized cut-off of 0.40 we found. Satisfactory analytical performance have also recently been reported for the Elecsys and LIAISON anti-SARS-CoV-2 assays and were not reassessed in this study. [15] [16] [17] [18] 28 4.1 | Specificity of the three automated assays 15 found a specificity of 100% and 99% using the manufacturer's cut-off or an adapted cut-off (i.e., >6.1 AU/ml), respectively. We found lower specificities of 97.9% and 91.5% using the manufacturer's cut-off or our adapted cut-off (i.e., >3.94 AU/ml). Plebani et al. found similar specificities of 96.8% and 88.9% using the manufacturer's cut-off or an adapted cut-off (i.e., >6.2 AU/ml).

The higher specificity observed in the study of Tré-Hardy et al. 15 is probably due to the lower number of samples included (n = 81) for the specificity calculation compared to our study (n = 141) and the one of Plebani et al. (n = 191) . 30 Interestingly, adapted cut-offs proposed on the LIAISON assay were all lower (>3.94, >6.1 AU/ml, 15 >6.2 AU/ml 30 ) than the manufacturer's cut-off (i.e., ≥12.0 AU/ml) using three independent cohorts of patients. The performance of these optimized cut-offs are not considered clinically different as there is an overlap between 95% confidence intervals.

Current data suggest that seroconversion occurs approximately 7-14 days after symptom onset. 6, 14, 31, 32 Although the Elecsys and the EUROIMMUN assays detected more positive results earlier after onset of symptoms than the LIAISON assay, none of the assays demonstrated a high enough clinical sensitivity to diagnose acute infection (i.e., <14 days). From 14 days since symptom onset, sensitivities increased for all assays, especially using optimized cut-offs. Using manufacturer's cutoffs resulted in 8, 10, and 7 false-negative specimens for the EUROIMMUN, LIAISON and Elecsys assays, respectively. Optimized cut-offs gave less false-negative results (4, 3, and 4, respectively). Due to the overlapping of confidence intervals at 95% between assays, we cannot conclude that one assay had a significantly higher true positivity rate.

Two studies having included less patients with symptoms since at least 14 days evaluated the performance of the Elecsys assay. 17, 18 Tang et al. found a sensitivity of 89.4% (n = 47) 17 and Egger et al. a sensitivity of 100% (n = 18). By using the manufacturer's cut-off, we found a somewhat similar sensitivity compared to Tang et al.

(i.e., 92.0%). However, they did not determine an optimized cut-off to increase the performance of the test. Fourteen days after RT-PCR positivity, Tré-Hardy et al. 15 found a sensitivity of 91% and 100%, F I G U R E 1 ROC curve analysis of three anti-SARS-CoV-2 immunoassays at more than 2 weeks after the symptom onset (n = 87) 6 | using the manufacturer's and an optimized cut-off on the LIAISON assay. It is important to note that with the confidence interval around 100% they found (92%-100%) was consistent with our results (i.e., 96.6% sensitivity; 95% CI, 90.3%-99.3%). Plebani et al. 30 published results in agreement with our finding with a sensitivity of 97.1% for the LIAISON assay.

All three anti-SARS-CoV-2 assays had equivalent sensitivities 14 days from symptom onset to diagnose past-COVID-19 infection. We also confirmed that anti-SARS-CoV-2 determination before Day 14 is of less clinical interest. However, the Elecsys assay had a higher specificity compared to the EUROIMMUN and the LIAISON assays.

Further studies specifically designed to evaluate long-term evolution of antibody response are also needed.

We wish to thank the personnel of the Saint-Luc Bouge laboratory for their technical assistance. 

The data that supports the findings of this study are available in the main manuscript and supplementary material of this article. The detailed clinical/biological/radiological data of COVID-19 confirmed patients are not publicly available due to privacy or ethical restrictions.

http://orcid.org/0000-0002-9805-049X

COVID-19-navigating the uncharted

World Health Organization. Coronavirus disease 2019 (COVID-19) Situation Report -129

In vitro diagnostic assays for COVID-19: recent advances and emerging trends

Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19)

The important role of serology for COVID-19 control

Analytical performances of a chemiluminescence immunoassay for SARS

SARS-CoV-2 serology: much hype, little data

Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease

The role of serology for COVID-19 control: population, kinetics and test performance do matter

COVID-19 and postinfection immunity: limited evidence, many remaining questions

Antibody tests for COVID-19: drawing attention to the importance of analytical specificity

Molecular, serological, and biochemical diagnosis and monitoring of COVID-19: IFCC taskforce evaluation of the latest evidence

Validation of a chemiluminescent assay for specific SARS-CoV-2 antibody

Clinical performance of the Elecsys electrochemiluminescent immunoassay for the detection of SARS-CoV-2 total antibodies

Clinical performance of the Roche SARS-CoV-2 serologic assay

Comparison of the Elecsys (R) Anti-SARS-CoV-2 immunoassay with the EDI enzyme linked immunosorbent assays for the detection of SARS-CoV-2 antibodies in human plasma

EUROIMMUN insert kit, anti-SARS-CoV-2 (IgG) nucleocapsid assay

User Verification of Precision and Estimation of Bias. Approved guideline. CLSI Document EP15-A3

Guide Technique d′Accréditation de Vérification (Portée A)/ Validation (Portée B) des Méthodes en Biologie Médicale -Document SH GTA 04 (révision 01)

Antibody responses to SARS-CoV-2 in patients with COVID-19

Clinical performance of two SARS-CoV-2 serologic assays

Evaluation of two automated and three rapid lateral flow immunoassays for the detection of anti-SARS-CoV-2 antibodies

Assessment of immune response to SARS-CoV-2 with fully automated MAGLUMI 2019-nCoV

Analytical and clinical validation of an ELISA for specific SARS-CoV-2 IgG, IgA, and IgM antibodies

High clinical performance and quantitative assessment of antibody kinetics using a dual recognition assay for the detection of SARS-CoV-2 IgM and IgG antibodies

Evaluation of the Roche Elecsys antio-SARS-CoV-2 Assay

Role of serology in the COVID-19 pandemic

Diagnostic performances and thresholds: the key to harmonization in serological SARS-CoV-2 assays?

Unexpected kinetics of anti-SARS-CoV-2 total antibodies in two patients with chronic lymphocytic leukemia

Response of anti-SARS-CoV-2 total antibodies to nucleocapsid antigen in COVID-19 patients: a longitudinal study