key: cord-0846582-d55uq2s9 authors: Hönemann, Mario; Lück, Christian; Maier, Melanie; Pietsch, Corinna; Dietze, Nadine; Berthold, Tom; Encalada, Marco Vinicio Narvaez; Grünewald, Thomas; Neumeister, Volker; Dalpke, Alexander; Liebert, Uwe Gerd title: Comprehensive evaluation of eight commercial SARS-CoV-2 IgG assays date: 2021-03-22 journal: Diagn Microbiol Infect Dis DOI: 10.1016/j.diagmicrobio.2021.115382 sha: 8c8ff6af46b22419aab1b0c0a061bf0aba764225 doc_id: 846582 cord_uid: d55uq2s9 Sensitivity and specificity of serological assays are key parameters for the accurate estimation of SARS-CoV-2 sero-prevalence. The aim of this study was to compare eight readily available IgG antibody tests using a panel of well-defined serum samples of pre-pandemic and pandemic origin. A cross-reaction panel included samples of patients with recent infection with either of the endemic Coronaviruses 229E, NL63, HKU1, or OC43. Additionally, samples with high antibody levels against influenza virus, adenovirus, and during acute EBV infection were included. Previous infection with endemic coronaviruses caused a significant amount of cross-reactivity in two of the assays. In contrast, the confidence intervals for the assays of Abbott, DiaSorin, Euroimmun and Roche encompassed the value of 98% for samples with a previous endemic hCoV infection. For all assays, sensitivities were between 91.3 and 98.8%. Assay performance was independent of the usage of either nucleocapsid or spike proteins. Since its emergence in December 2019 in the Chinese province of Hubei 1 , the novel coronavirus SARS-CoV-2 caused significant mortality and morbidity leading to more than 100 million proven infections and 2.300.000 deaths in more than 200 countries world-wide 2,3 . Coronaviruses are enveloped and have a large positive-sense, single-stranded RNA-genome 4 . The disease resulting from SARS-CoV-2 infection, COVID-19, is mild in the majority of cases. However, infection may also result in severe disease courses with acute respiratory distress, multi-organ failure, and death 2 . In acute infections, diagnosis is confirmed by nucleic acid amplification testing (NAAT) of nasal or oropharyngeal swabs as well as deep respiratory specimen. Specific antibodies appear in sufficient amounts not earlier than 10 to 14 days after infection 5, 6 . Asymptomatic infections as well as patients having rather mild symptoms may result in underestimating the total disease burden. Although ongoing testing for acute infections will provide a point prevalence of acute infections, false negative PCR results, i.e. due to inadequate sampling or low viral load, should be taken into account. The detection of specific IgG directed against SARS-CoV-2 allows the identification of previously infected persons and patients and thus helps to establish more precisely the true prevalence of SARS-CoV-2 in a population. High specificity and sensitivity of serological tests are of paramount importance. Different antigenic sites on the structural proteins show a varying degree of homology to other members of the Orthocoronavirinae, which could cause false positive test results. Back in 2002/2003, cross-reactivity was noted in the SARS epidemic 7 . Notably, four other human Coronaviruses (HCoV), namely the alpha-coronaviruses HCoV-229E and HCoV-NL63 and the beta-coronaviruses HCoV-OC43 and HCoV-HKU1, circulate in the human population 8 . The endemic human coronaviruses are widespread causing predominantly mild infections of the upper respiratory tract 9, 10 . Infections could give rise to cross-reactive antibodies depending on their affinity to the specific antigen and thus may influence the performance of SARS-CoV-2 serological assays. The aim of the study was to compare different commercial assays on a panel of selected serum samples with a focus on an in-depth analysis of specificity and cross-reactivity. Anti-SARS-CoV-2-ELISA (IgG) and (5) Anti-SARS-CoV-2-NCP-ELISA (IgG) (Euroimmun, Lübeck, Germany), (6) recomWell SARS-CoV-2 IgG (Mikrogen GmbH, Neuried, Germany), (7) COVID-19 ELISA IgG (Vircell, Granada, Spain), (8) Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland). Assay specifications are provided in the manufacturer's manuals (table 1) . In total, 257 samples of 222 individual patients were included into the analysis. Specificity analysis. The majority of antibody assays delivered negative test results. However, sero-reactivity in the pre-pandemic samples differed considerably between the commercial assays (table 2) . Lowest specificities were seen for the hCoV NL63 subgroup (Epitope Diagnostics) and the hCoV HKU1 subgroup (Vircell). Reactive samples were mostly reactive in one of the assays, except for six samples that reacted in both the assays of Epitope Diagnostics and Mikrogen. The confidence interval for the assays of Abbott, DiaSorin, Euroimmun, and Roche encompassed a value of at least 98% for the samples with a known previous endemic hCoV infection. While high levels of antibodies against influenza viruses A and B and adenovirus were only associated with an increased cross-reactivity in one assay (Epitope Diagnostics), EBV infection was associated with specificities below 70% for two assays (Epitope diagnostics and Vircell). Again, reactive samples were mostly reactive in one of the assays with exception of one influenza sample that reacted in four of the assays (Epitope diagnostics, Abbott, Mikrogen, and Euroimmun (NCP)). Two other samples were reactive in the assays of Epitope diagnostics, Mikrogen, and Euroimmun (NCP). For pandemic samples ( for each of the used assays (figure 1). Spearman-Rho correlation coefficients were highest for the assays of Euroimmun (S1) and Diasorin (ρ=0.84), and Euroimmun (NCP) and Abbott (ρ=0.81). Antibody tests are generally suitable (1) to determine seroprevalence in a cohort of patients to be used for defining a denominator of the overall magnitude of regional past infections, (2) to identify blood donors which may serve for the preparation of convalescent plasma, and (3) to investigate rare complications of a SARS-CoV-2 infection, e.g. in children 14 . A pivotal quality of a serological test is high specificity in order to exclude false positive results. However, large discrepancies to the manufacturers' specifications and to previous studies were determined for some of the assays when challenged with potential crossreactive samples. In line with previous studies, the highest specificities were seen for the assays of Roche and Abbott, which were reported to be above 99% [15] [16] [17] . Both were the most reliable assays in the cross-reaction panel. Of the two Euroimmun assays, the one using the spike protein (S1) was previously reported with specificities between 96 and 100% 16, 18, 19 . Interestingly, the alpha-coronaviruses HCoV-229E and HCoV-NL63 appeared to have a similar impact on cross-reactivity, in spite of the fact that SARS-CoV-2 is a member of betacoronaviruses. However, this could be a random effect of the small sample size for the individual coronavirus subgroups. Depending on the rate of coronavirus infections in a respiratory season and occurrence in a given geographical area, the positive predictive values of a given serological SARS-CoV-2 assay may therefore be influenced to varying extents. Performance of the assays was test specific and not dependent on the used antigen as assays using either the nucleocapsid or spike-protein (i.e. both Euroimmun assays) were able to demonstrate an equally high specificity. An acute infection with EBV did not lead to cross-reactivation except in two assays (Epitope Diagnostics and Vircell). Additionally, the presence of high levels of antibodies against Influenza viruses A and B and Adenovirus were only associated with a marked cross-reactivation in one of the assays (Epitope Diagnostics). The specificity of the assays was closest to the ones stated by the manufacturers when randomly selected samples from the diagnostic routine were used, equivalent to test validations using random blood donor samples. However, we show that a cross-reaction panel is necessary to thoroughly characterize the performance of a serological assay. The sensitivity of all investigated assays was estimated to be above 90%. Especially, when only samples after 15 days of symptom onset were considered, the sensitivity was similar between the different assays. Additionally, based on an assumed 5% prevalence of SARS-CoV-2 infected individuals, the negative predictive value of all assays was above 99%. The highest sensitivities were calculated for the assays of Euroimmun (NCP) and Vircell. The comparison to other studies, however, is difficult due to the wide spectrum of collection time points after symptom onset. Means to increase sensitivity, may include the use of two assays as reported previously 32 . However, these results underline the recommendation by the World Health Organization not to apply serological assays for the determination of acute infections 33 . Spearman-Rho correlation coefficients were moderate for most of the compared assay combinations. The highest coefficient was detected for the S-based assays of Diasorin and Euroimmun, which was also reported before 17 , followed by the N-based assays of Abbott and Euroimmun. The best correlation of two assays using different proteins was calculated for both assays of Euroimmun. Interestingly, the assay of Roche only showed a single correlation that was above 0.5 with one of the other assays (Abbott). It is the only pan-Ig assay included in this study and thus, this may be due to an unknown proportion of IgA and IgM reactivity of the assay result. This may be an indication for different areas of application for different test systems: assays for the detection of (formerly infected) individuals and assays that allow the estimation of the level of the immune response after infection or upon vaccination 19, 34 . For the later, S1-based assays may be most suitable since the neutralizing immunity is directed towards the spike protein. Interassay correlation may be a requirement for the establishment of international standards. However, correlation of a positive result to immunity is still not well understood. In order to infer humoral immunity, a virus neutralization assay needs to be applied, but a variable degree of correlation was reported 27, 35, 36 . Furthermore, even neutralization tests may not allow a reliable prediction of the immunity against SARS-CoV-2 and its duration. There are several limitations of this study. The seroprevalence for each of the endemic coronaviruses did not reach 100% in the preselected samples. Unfortunately, there is a lack of thoroughly validated test systems for the detection of the serological response towards endemic coronaviruses, which is also acknowledged by the manufacturer of the used assay. Thus, the results need to be interpreted with caution as neither false negative results nor cross-reactivity can be ruled out. However, the time frame of samples after the infection with an endemic coronavirus used in this study is similar to ones described before 5 . Additionally, no seroconversion panel was included in this study to adequately describe the sensitivity in different periods after symptom onset. Nevertheless, the analysis of eighty samples allowed a valuable comparison of the investigated assays. In conclusion, high specificity is of paramount importance and should be the main criteria for the choice of a respective test system. The investigated assays by Abbott, DiaSorin, Euroimmun, and Roche showed the highest estimated specificity, especially with regard to other human Coronaviruses. Other assays may be applied with a subsequent validation by the aforementioned tests. Specificity is provided as percent for the negative samples in total and stratified by subgroups. (CI, 95% confidence interval; PPV, positive predictive value) The seroprevalence of cross-reactive samples (A) was characterized for the indicated pathogens. For EBV the amount of acute infections is given. For Influenza virus A and B and adenovirus the mean reactivity is given. All parameters are also shown for the samples which were specifically selected for the respective pathogen (B) and for the samples of the pandemic samples of SARS-CoV-2 NAAT negative patients of 2020 (C). (P, seroprevalence;* reactive for at least one endemic coronavirus; ** reactive for at least two endemic coronaviruses) A Novel Coronavirus from Patients with Pneumonia in China The SARS-CoV-2 outbreak: What we know Coronaviurs Disease (COVID-19) Situation Report Origin and evolution of pathogenic coronaviruses Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Responses in Coronavirus Disease The dynamics of humoral immune responses following SARS-CoV-2 infection and the potential for reinfection Antigenic cross-reactivity between severe acute respiratory syndrome-associated coronavirus and human coronaviruses 229E and OC43 Genetic Recombination, and Pathogenesis of Coronaviruses Clinical impact of human coronaviruses 229E and OC43 infection in diverse adult populations Epidemiology and clinical presentations of the four human coronaviruses 229E, HKU1, NL63, and OC43 detected over 3 years using a novel multiplex realtime PCR method Detection of 2019 novel coronavirus (2019-nCoV) by realtime RT-PCR R: A language and environment for statistical computing. R Foundation for Statistical Computing GGally: Extension to 'ggplot2 Kawasaki-like disease: emerging complication during the COVID-19 pandemic. The Lancet Performance Characteristics of the Abbott Architect SARS-CoV-2 IgG Assay and Seroprevalence in Review of Current Advances in Serologic Testing for COVID-19 Clinical evaluation of five different automated SARS-CoV-2 serology assays in a cohort of hospitalized COVID-19 patients Comparison of test performance of commercial anti-SARS-CoV-2 immunoassays in serum and plasma samples Brief clinical evaluation of six high-throughput SARS-CoV-2 IgG antibody assays Evaluation of commercial and automated SARS-CoV-2 IgG and IgA ELISAs using coronavirus disease (COVID-19) patient samples Humoral Immune Response to SARS-CoV-2 Comparison of four new commercial serologic assays for determination of SARS-CoV-2 IgG Performance evaluation of serological assays to determine the immunoglobulin status in SARS-CoV-2 infected patients Test performance evaluation of SARS-CoV-2 serological assays Evaluation of the EDI enzyme linked immunosorbent assays for the detection of SARS-CoV-2 IgM and IgG antibodies in human plasma Enzyme-Linked Immunosorbent Assay for the Detection of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) IgM/IgA and IgG Antibodies Among Healthcare Workers Clinical and Analytical Performance of an Automated Serological Test That Identifies S1/S2-Neutralizing IgG in COVID-19 Patients Semiquantitatively Characterization of spike glycoprotein of SARS-CoV-2 on virus entry and its immune cross-reactivity with SARS-CoV Review of Viral Testing (Polymerase Chain Reaction) and Antibody/Serology Testing for Severe Acute Respiratory Syndrome-Coronavirus-2 for the Intensivist The nucleocapsid protein of human coronavirus NL63 Genomic characterisation and epidemiology of 2019 novel coronavirus: implications for virus origins and receptor binding Spread of SARS-CoV-2 in the Icelandic Population Diagnostic testing for SARS-CoV-2 (last accessd Quantification of SARS-CoV-2 antibodies with eight commercially available immunoassays A highly specific and sensitive serological assay detects SARS-CoV-2 antibody levels in COVID-19 patients that correlate with neutralization Antibody seroconversion in asymptomatic and symptomatic patients infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)