key: cord-0878242-fv7d5j0h
authors: Marlet, Julien; Petillon, Camille; Ragot, Emma; Abou El Fattah, Yazid; Guillon, Antoine; Marchand Adam, Sylvain; Lemaignen, Adrien; Bernard, Louis; Desoubeaux, Guillaume; Blasco, Hélène; Barin, Francis; Stefic, Karl; Gaudy-Graffin, Catherine
title: Clinical performance of four immunoassays for antibodies to SARS-CoV-2, including a prospective analysis for the diagnosis of COVID-19 in a real-life routine care setting
date: 2020-09-08
journal: J Clin Virol
DOI: 10.1016/j.jcv.2020.104633
sha: ab5788c587ba67f25af2a629952a56a13816314e
doc_id: 878242
cord_uid: fv7d5j0h

OBJECTIVES: The aim of the present study was to evaluate the clinical performance of four SARS-CoV-2 immunoassays and their contribution in routine care for the diagnosis of COVID-19, in order to benefit of robust data before their extensive use. METHODS: The clinical performance of Euroimmun ELISA SARS-CoV-2 IgG, Abbott SARS-CoV-2 IgG, Wantai SARS-CoV-2 Ab ELISA, and DiaPro COVID-19 IgG confirmation were evaluated in the context of both a retrospective and a prospective analysis of COVID-19 patients. The retrospective analysis included plasma samples from 63 COVID-19 patients and 89 control (pre-pandemic) patients. The prospective study included 203 patients who tested either negative (n = 181) or positive (n = 22) by RT-PCR before serology sampling. RESULTS: The specificity was 92.1 %, 98.9 %, 100 % and 98.9 % and the sensitivity 14 days after onset of symptoms was 95.6 %, 95.6 %, 97.8 % and 95.6 % for Euroimmun IgG, Abbott IgG, Wantai Ab, and DiaPro IgG confirmation SARS-CoV-2 immunoassays, respectively. The low specificity of Euroimmun IgG (for ratio <5) was not confirmed in routine care setting (98.5 % negative agreement). Serology was complementary to RT-PCR in routine care and lead to identification of false positive (Ct>38, <2 targets detected) and false negative RT-PCR results (>1 month post onset of symptoms). CONCLUSIONS: Serology was complementary to RT-PCR for the diagnosis of COVID-19 at least 14 days after onset of symptoms. First line serology testing can be performed with Wantai Ab or Abbott IgG assays, while DiaPro IgG confirmation assay can be used as an efficient confirmation assay.

The novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) firstly reported in late 2019 in Wuhan [1] and causing coronavirus disease has spread across the world and lead to a worldwide sanitary crisis. Detection of viral RNA using reverse-transcriptase polymerase chain reaction (RT-PCR) in respiratory samples is the gold standard for early diagnosis of COVID-19. However, sensitivity of this molecular diagnosis starts to decrease at week 3 after onset of symptoms [2] . Complementary to RT-PCR in respiratory samples, SARS-CoV-2 serology allows identification of COVID-19 cases with a higher sensitivity than RT-PCR several days after onset of symptoms [3] . In addition, it can be used to determine the fraction of the population that has been exposed to the virus [3] . However, results of such serosurveys depend on the performance of immunoassays and on the seroprevalence of SARS-CoV-2 which remains quite low, even in COVID-19 hotspots [4] . Given this low prevalence, it is crucial to have robust data evaluating those assays before clinical or epidemiological use. Four immunoassays were evaluated in our study: Euroimmun ELISA SARS-CoV-2 IgG (Euroimmun, Lübeck, Germany), Abbott SARS-CoV-2 IgG (Abbott Diagnostics, Illinois, USA), Wantai SARS-CoV-2 Ab ELISA (Beijing Wantai Biological Pharmacy Enterprise, Beijing, China), and DiaPro COVID-19 IgG Confirmation (Diagnostic Bioprobes, Milano, Italy). The latter assay was used to determine the specificity of SARS-CoV-2 Ab against S1, S2 and N Ag. The first aim was to evaluate the performance of these SARS-CoV-2 immunoassays on a series of 63 COVID-19 patients and 89 pre-pandemic control patients. The second aim was to evaluate their contribution in routine care to confirm or infirm the diagnosis of COVID-19.

Two complementary studies were performed. First, clinical performance of immunoassays were evaluated on 63 COVID-19 patients tested positive for SARS-CoV-2 RNA by RT-PCR at Tours University Hospital (Table 1) . SARS-CoV-2 RT-PCR were performed in respiratory samples using Allplex™ 2019-nCOV assay (Seegene, Seoul, Republic of Korea), Abbott RealTime SARS-CoV-2 assay (Abbott Molecular, Illinois, USA) or Bosphore 2019-nCoV detection kit (Anatolia GeneWorks, Istanbul, Turkey) depending on reagents and systems availability. Among the positive RT-PCR results, inconclusive RT-PCR results were defined as results positive only for one gene (E, ORF1ab or N). All 63 patients required an in-patient hospital stay for COVID-19 and had plasma samples collected between April 8th and May 11th 2020. Retrospective testing for SARS-CoV-2 Ab was performed on these samples, collected between 2-36 days after the onset of symptoms. Mild and critical COVID-19 cases were defined according to WHO [31] . Specificity was evaluated on plasma collected before the end of 2019 in 89 patients from occupational medicine (n = 30), emergency or pneumology departments (n = 26) or from patients tested positive by RT-PCR (Allplex™ RP3, Seegene) for seasonal coronaviruses (n = 33, OC43, 229E or NL63) between 3-82 weeks before serology sampling. Positive and negative predictive values of immunoassays were estimated in a context of low (2.4 %) and high seroprevalence (9.8 %) of SARS-CoV-2 Ab. These estimates were based on prevalence of SARS-CoV-2 Ab in healthcare professionals (2.4 %, 108/4 444 tested from June to August 2020) or from patients (9.8 %, 6/61 tested from May to June 2020).

Second, the contribution of SARS-CoV-2 serology in routine care for the diagnosis of COVID-19 was evaluated on 203 patients tested for SARS-CoV-2 infection by RT-PCR between April 8th and June 11th 2020 ( Table 1) . Most of these patients were healthcare professionals (167/ 203, 82.3 %) who did not require an in-patient hospital stay (125/167, 74.9 %). Other patients required an in-patient hospital stay (31/203) or had ambulatory testing (6/203 ). These patients were tested for SARS-CoV-2 antibodies at least 14 days after RT-PCR testing (unless otherwise specified) between June 1st and June 25th 2020. The entire study was performed according to French Reference Methodology MR-004, after patient information and anonymization of data. Samples were obtained from the registered biological collection DC-2020− 3961.

Euroimmun ELISA SARS-CoV-2 IgG, Abbott SARS-CoV-2 IgG (Alinity-I analyzer), Wantai SARS-CoV-2 Ab ELISA, and DiaPro COVID-19 IgG Confirmation assays were performed according to the manufacturer's recommendations. Euroimmun IgG assay was used as first line immunoassay in routine care setting. Positive or undetermined results and results discordant with RT-PCR were confirmed with other assays. For statistical analysis, Euroimmun IgG and Wantai Ab uninterpretable results were considered negative. DiaPro IgG confirmation assay was considered positive when Ab against at least two targets (S1, S2 or nucleoprotein) were detected.

Statistical analysis was carried out using Graphpad Prism v5. Comparison of sensitivity and specificity were performed using McNemar's test. All tests were two-sided at the 0.05 significance level.

The sensitivity of Euroimmun IgG, Abbott IgG, Wantai Ab, and DiaPro IgG confirmation SARS-CoV-2 assays seven to thirteen days after the onset of symptoms were 30.8, 46.2, 84.6 and 61.5 % ( Fig. 1 and Table 2 ). In this timeframe, the DiaPro IgG confirmation demonstrated an excellent sensitivity for anti-N Ab (100 %), higher than that for anti-S2 Ab (15.4 %, p = 0.003) and higher, although not significantly, than that for anti-S1 Ab (53.8 %; p = 0.13).

The sensitivity of Euroimmun IgG, Abbott IgG, Wantai Ab and DiaPro IgG confirmation SARS-CoV-2 assays 14 days after the onset of symptoms were 95.6 %, 95.6 %, 97.8 % and 95.6 %, respectively ( Fig. 1 and Table 2 ). The DiaPro IgG confirmation assay demonstrated good and similar sensitivities for anti-S1 and anti-N Ab (93.3 % and 97.8 %), both higher than that for anti-S2 Ab (62.2 %, p ≤ 0.002). The single patient who tested negative with all immunoassays was a 61 years old heart transplant patient who experienced fever and myocarditis and had inconclusive RT-PCR result (positive only for the N gene) 26 days postonset of symptoms (Fig. 1) . This patient could have had a false positive RT-PCR result or, less likely false negative serology results because of its immunosuppressive treatments.

The specificity of Euroimmun IgG, Abbott IgG, Wantai Ab and DiaPro IgG confirmation SARS-CoV-2 immunoassay were 92.1 %, 98.9 %, 100 % and 98.9 %, respectively ( Fig. 1 and Table 2 ). Specificity of the DiaPro IgG confirmation assay was lower for anti-N Ab (84.3 %) than for anti-S1 Ab (95.5 %, p = 0.02) and anti-S2 Ab (100.0 %, p = 0.0005). Specificity of Euroimmun IgG assay was lower than other immunoassays, with a significant difference versus Wantai Ab assay (p = 0.02) but not versus Abbott IgG (p = 0.08). False positive Euroimmun IgG results were observed in in an equivalent manner in the different groups of control patients: those with seasonal coronaviruses infections (2/33, both OC43), those from emergency or pneumology departments (3/26) and those from occupational medicine (2/30). False positive Euroimmun IgG results in control patients were associated with a lower ratio (median of 3.13; IQR 1.90-4.40, maximum 5) than those from the COVID-19 patients (median of 7.60, IQR 3.20-11.14, p = 0.02) (Fig. 1) 

Agreement between RT-PCR and Euroimmun IgG was 68 % (15/22) and 96 % (173/181) for patients who tested positive and negative by RT-PCR, respectively. Results of other immunoassays (Fig. 2) , the delay between RT-PCR and serology as well as Ct and targets of RT-PCR results were analyzed for these patients (Fig. 3) . Patients were considered as suffering from COVID-19 if they tested positive with RT-PCR (all targets positive with Ct<38) and/or with at least two out of four SARS-COV-2 immunoassays (Fig. 2) . This allowed an accurate definition of 23 We considered that false positive RT-PCR results occurred in four patients. They were characterized by inconclusive RT-PCR results. Only the N gene was detected, with Ct>38. These patients tested negative with all immunoassays, suggesting the absence of exposure to SARS-CoV-2. Importantly, such inconclusive RT-PCR results were also observed in two COVID-19 patients confirmed positive with all serological assays (Fig. 2) , precluding any systematic interpretation of inconclusive RT-PCR results as false positive.

False negative RT-PCR results were observed in two asymptomatic patients, and in three symptomatic patients with late RT-PCR sampling (>1 month after onset of symptoms). These three symptomatic individuals were positive with all immunoassays, confirming the benefit of serology testing for patients with late presentation after onset of symptoms. False negative serology results were observed in 2/23 COVID-19 patients who were sampled too early for serology testing (<14 days after onset of symptoms, Figs. 2 and 3) . Among the 21 remaining COVID-19 patients tested for antibody to SARS-CoV-2 at least 14 days post onset of symptoms, positive agreement with COVID-19 diagnosis was 90.5 % (19/21), 76.2 % (16/21), 90.5 % (19/21) and 95.2 % (20/21) for Euroimmun IgG, Abbott IgG, Wantai Ab and DiaPro IgG confirmation assay, respectively. The DiaPro IgG confirmation assay had the highest positive agreement with COVID-19 diagnosis. It detected anti-S1, -S2 and -N Ab in 20, 12 and 21 of these 21 patients, respectively. In contrast, 5 out of 21 COVID-19 patients were not detected with the Abbott IgG assay. Interestingly, these five COVID-19 patients were also negative for anti-S2 Ab with the DiaPro IgG confirmation assay (Fig. 2) . The low positive agreement of Abbott IgG (16/21) with COVID-19 confirmed cases might suggest a lower sensitivity. However, the data could be due to the small sample size and would deserve to be confirmed in larger studies.

In the first part of our analysis corresponding to the retrospective evaluation, the specificity was 92.1 %, 98.9 %, 100 %, and 98.9 % for Euroimmun IgG, Abbott IgG, Wantai Ab, and DiaPro IgG confirmation SARS-CoV-2 assays, respectively. The sensitivity 14 days after onset of symptoms was 95.6 %, 95.6 %, 97.8 %, and 95.6 % for Euroimmun IgG, Abbott IgG, Wantai Ab, and DiaPro IgG confirmation SARS-CoV-2 assays, respectively. The sensitivity between 7 and 13 days was suboptimal for Wantai Ab (84.6 %) and inadequate (<75 %) for other assays. Our results are in accordance with the available data for which the sensitivity ranges 14 days after onset of symptoms have been described as 61.7-96.0 %, 77.8-100.0 %, 98-100 % and specificity ranges as 86.6-100.0 %, 95.1-100.0 %, 98.0-99.1 % for Euroimmun IgG [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] , Abbott IgG [9, 10, 13, [19] [20] [21] and Wantai Ab immunoassays [15, 22, 23] , respectively. Although we included a collection of serum samples from patients for whom a recent infection by seasonal coronaviruses was documented, we did not notice any specific clustering of false positive results that could be attributed to a particular cross-reactivity.

Clinical performance of the DiaPro IgG confirmation assay (combining S1, S2 and N Ag) were similar to other immunoassays and to the manufacturer's statement (sensitivity 98 %, specificity 90 %). Interestingly, a higher sensitivity was observed for anti-N Ab and anti-S1 Ab than for anti-S2 Ab (97.8 and 93.3 vs 62.2 %, p ≤ 0.002). This confirmed previous studies based on in-house ELISAs [24] . In contrast, a higher specificity was observed for anti-S2 Ab (100 %) than for anti-N Ab (84.3 %, p = 0.0005). As suggested by previous studies [25, 26] , combination of S and N Ag probably contributed to the overall good performances of this assay, which had the highest positive agreement of all immunoassays (20/21) with COVID-19 diagnosis in our sub-study performed in real-life routine conditions. Furthermore, this assay has the advantage to allow comparison of the ratio between anti-N IgG and anti-S IgG, which has been associated with the prognosis [26] .

We observed a lack of specificity for the Euroimmun IgG assay (92.1 %, CI 95 %: 84.5− 96.8 %) in comparison to other assays in our retrospective study. This low specificity was not so problematic in the routine care setting since 98.5 % of the non-COVID-19 patients tested negative with Euroimmun IgG. There are conflicting data in the literature regarding the specificity of this assay. It demonstrated a good specificity (≥ 95 %) in most studies [7, [10] [11] [12] 14, 18, 19] , while a minority of studies suggested otherwise [6, 9, 17] . In this context and given the low prevalence of SARS-CoV-2 Ab, confirmation of positive Euroimmun IgG results seems reasonable. RT-PCR also demonstrated a relative lack of specificity in routine care setting (negative agreement of 97.8 %, 176/180) especially for inconclusive RT-PCR results (only one gene detected, with Ct>38).

Some false negative serology results were associated with early serology sampling (<14 dps), while all false negative RT-PCR results were associated with late RT-PCR sampling (>30dps). This confirms that timing of testing is critical for good sensitivity of SARS-CoV-2 RT-PCR and serology [3] . However, early serology sampling does not explain the relative lack of agreement with COVID-19 diagnosis for Euroimmun IgG (19/21), Abbott IgG (16/21), Wantai Ab (19/21), and DiaPro IgG confirmation assay (20/21) in routine care setting. Similar observations have been made in populations of healthcare professionals, with false negative rates ranging from 1 % [27] to 20 % [28] [29] [30] several weeks after disease. This could be due to the high proportion of patients with mild COVID-19, resulting in low rates of seroconversion [28] .

Our study confirms that Abbott IgG, Wantai Ab and DiaPro IgG confirmation assays are suitable assays for the diagnosis of COVID-19 at least 14 days after onset of symptoms. Main advantages of these assays are their automation (Abbott IgG), their optimal clinical performance (Wantai Ab) and their ability to differentiate between anti-N, -S1 and -S2 Ab (DiaPro IgG confirmation). DiaPro IgG confirmation assay has a low throughput (four wells per patient) which is adequate for confirmation testing. Euroimmun IgG assay can also be used for the diagnosis of COVID-19 if positive results are confirmed with another assay to compensate for its low specificity.

The huge impact of the SARS-CoV-2 emergence in public health justifies extensive seroepidemiological studies to survey its spread in various populations and numerous settings. There is a burst of serologic assays rolling out in different formats, including simple rapid tests. Our study shows that specificity may be highly variable among available immunoassays for antibody to SARS-CoV-2. Poor specificity of an assay in a population where prevalence and incidence of COVID-19 are low will lead to irrelevant data. Our study, as others, stresses on the absolute necessity to use only carefully validated assays to provide epidemiological data useful to public health decision makers.

China novel coronavirus investigating and research team

Virological assessment of hospitalized patients with COVID-2019

S147330992030517X

SARS-CoV-2 seroprevalence in COVID-19 hotspots

Evaluation of nine commercial SARS-CoV-2 immunoassays

Diagnostic performance of seven rapid IgG/IgM antibody tests and the Euroimmun IgA/IgG ELISA in COVID-19 patients

Evaluation of the EUROIMMUN Anti-SARS-CoV-2 ELISA assay for detection of IgA and IgG antibodies

Performance of six SARS-CoV-2 immunoassays in comparison with microneutralisation

Assessment of SARS-CoV-2 serological tests for the diagnosis of COVID-19 through the evaluation of three immunoassays: two automated immunoassays (Euroimmun and Abbott) and one rapid lateral flow immunoassay (NG Biotech)

Clinical performance of different SARS-CoV-2 IgG antibody tests

Evaluation of two automated and three rapid lateral flow immunoassays for the detection of anti-SARS-CoV-2 antibodies

Brief clinical evaluation of six high-throughput SARS-CoV-2 IgG antibody assays

Comparison of four new commercial serologic assays for determination of SARS-CoV-2 IgG

Quantification of SARS-CoV-2 antibodies with eight commercially available immunoassays

Geneva center for emerging viral diseases, validation of a commercially available SARS-CoV-2 serological immunoassay

Detection of SARS-CoV-2 antibodies using commercial assays and seroconversion patterns in hospitalized patients

An evaluation of COVID-19 serological assays informs future diagnostics and exposure assessment

Performance characteristics of four high-throughput immunoassays for detection of IgG antibodies against SARS-CoV-2

Clinical evaluation of serological IgG antibody response on the Abbott Architect for established SARS-CoV-2 infection

INMICovid-19 laboratory team, performance evaluation of Abbott ARCHITECT SARS-CoV-2 IgG immunoassay in comparison with indirect immunofluorescence and virus microneutralization test

Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease

Comparison of diagnostic accuracies of rapid serological tests and ELISA to molecular diagnostics in patients with suspected coronavirus disease 2019 presenting to the hospital

Anti-spike, anti-nucleocapsid and neutralizing antibodies in SARS-CoV-2 inpatients and asymptomatic carriers

Comparison of the diagnostic sensitivity of SARS-CoV-2 nucleoprotein and glycoprotein-based antibody tests

Kinetics of SARS-CoV-2 specific IgM and IgG responses in COVID-19 patients

Serologic responses to SARS-CoV-2 infection among hospital staff with mild disease in eastern France

Differences in antibody kinetics and functionality between severe and mild SARS-CoV-2 infections

Point-of-care serological assays for delayed SARS-CoV-2 case identification among health-care workers in the UK: a prospective multicentre cohort study

Symptoms and immunoglobulin development in hospital staff exposed to a SARS-CoV-2 outbreak

The authors thanks Brigitte Berthon, Pascale Mezieres and Amélie Grivot for their contribution to the experiments, and Léo Léger for its contribution to data collection and analysis. Dr. Marchand-Adam reports financial relationships from Boehringer Ingelheim, Roche and Novartis outside the submitted work. Dr. Lemaignen reports financial relationships from Gilead, Pfizer and MSD outside the submitted work.