key: cord-338923-hc7gagnq authors: Jääskeläinen, AJ; Kuivanen, S; Kekäläinen, E; Ahava, MJ; Loginov, R; Kallio-Kokko, H; Vapalahti, O; Jarva, H; Kurkela, S; Lappalainen, M title: Performance of six SARS-CoV-2 immunoassays in comparison with microneutralisation date: 2020-06-15 journal: J Clin Virol DOI: 10.1016/j.jcv.2020.104512 sha: doc_id: 338923 cord_uid: hc7gagnq There is an urgent need for reliable high-throughput serological assays for the management of the ongoing COVID-19 pandemic. Preferably, the performance of serological tests for a novel virus should be determined with clinical specimens against a gold standard, i.e. virus neutralisation. We compared the performance of six commercial immunoassays for the detection of SARS-CoV-2 IgG, IgA and IgM antibodies, including four automated assays [Abbott SARS-COV-2 IgG (CE marked), Diasorin Liaison® SARS-CoV-2 S1/S2 IgG (research use only, RUO), and Euroimmun SARS-CoV-2 IgG and IgA (CE marked)], and two rapid lateral flow (immunocromatographic) tests [Acro Biotech 2019-nCoV IgG/IgM (CE marked) and Xiamen Biotime Biotechnology SARS-CoV-2 IgG/IgM (CE marked)] with a microneutralisation test (MNT). Two specimen panels from serum samples sent to Helsinki University Hospital Laboratory (HUSLAB) were compiled: the patient panel (N=70) included sera from PCR confirmed COVID-19 patients, and the negative panel (N=81) included sera sent for screening of autoimmune diseases and respiratory virus antibodies in 2018 and 2019. The MNT was carried out for all COVID-19 samples (70 serum samples, 62 individuals) and for 53 samples from the negative panel. Forty-one out of 62 COVID-19 patients showed neutralising antibodies.The specificity and sensitivity values of the commercial tests against MNT, respectively, were as follows: 95.1%/80.5% (Abbott Architect SARS-CoV-2 IgG), 94.9%/43.8% (Diasorin Liaison SARS-CoV-2 IgG; RUO), 68.3%/87.8% (Euroimmun SARS-CoV-2 IgA), 86.6%/70.7% (Euroimmun SARS-CoV-2 IgG), 74.4%/56.1% (Acro 2019-nCoV IgG), 69.5%/46.3% (Acro 2019-nCoV IgM), 97.5%/71.9% (Xiamen Biotime SARS-CoV-2 IgG), and 88.8%/81.3% (Xiamen Biotime SARS-CoV-2 IgM). This study shows variable performance values. Laboratories should carefully consider their testing process, such as a two-tier approach, in order to optimize the overall performance of SARS- CoV-2 serodiagnostics. Serosurveys are considered essential for creating timely snapshots for global and regional public health management of the ongoing COVID-19 pandemic [1] . Thus, there is an urgent need for the J o u r n a l P r e -p r o o f development of high-throughput serological assays, which enable population screening, as well as other epidemiological investigations. Setting up a serological assay for a completely novel pathogen is challenging in many respects. At present, there is inadequate knowledge as to when and what kind of immune response follows SARS-CoV-2 infection [2] . We are also yet to learn about factors that may disturb reliable serology, such as potential cross reaction from seasonal coronaviruses. The patient samples consisted of serum specimens sent to the Department of Virology and Immunology, Helsinki University Hospital Laboratory, Finland for diagnostic purposes. A subset of these specimens has been included in a previous publication evaluating the Euroimmun SARS-CoV-2 IgG and IgA assays, and are included here for comparison [3] . The negative panel consisted of 81 serum samples (from 81 individuals) (median age 64 years, range 2-89 years; 33 males, 48 females) ( Table 1) The analysis of SARS-CoV-2 IgG or IgA antibodies were carried out using the Architect Plus i2000sr Analyzer (Abbott, Illinois, USA) and SARS-COV-2 IgG CMIA kit (nucleoprotein based antigen; Abbott; CE marked), EUROLabworkstation (Euroimmun, Lübeck, Germany) and SARS-CoV-2 IgG and IgA ELISA kits (S1-based antigen; Euroimmun) and Diasorin Liaison ® XL (DiaSorin, Saluggia, Italy) and SARS-CoV-2 S1/S2 IgG CLIA kit (S1/S2 based antigen; DiaSorin; RUO) according to the manufacturers´ instructions. All samples from the negative panel (N=81) and the patient panel (N=70) were tested with Abbott SARS-CoV-2 IgG, and Euroimmun SARS-CoV-2 IgA and IgG. All samples from the negative panel (N=81) and (due to limited kit supply) 61/70 samples (53/62 individuals) from the COVID-19 patient panel were tested with DiaSorin SARS-CoV-2 S1/S2 IgG. MNT was performed in a BSL-3 laboratory as described previously [5] The growth medium was removed and the virus-serum mixture was added to the cells and incubated for 4 days at 37 °C with 5% CO2, after which the cells were stained with crystal violet to detect cytopathic effect (CPE). The neutralisation endpoint titer was determined as the endpoint serum dilution that inhibited the SARS-CoV-2 induced CPE in at least 2 out 3 parallel wells. The MNT titer ≥40 was considered as positive. For 55 COVID-19 patients out of 62, the date of disease onset was available, and disease severity could be rated (mild, moderate or severe; based on Siddiqi et al. [2020; 6] ) (Table 2, Figure 1 ). In the COVID-19 patients included in this study, the earliest time point for the MNT to become J o u r n a l P r e -p r o o f positive was 3 days from onset of illness (patient ID 50), while the furthest time point for a negative MNT was 16 days from onset (patient ID 5) ( Figure 1E ; Table 2 ). Disease severity did not appear to be reflected in the MNT titers of the patients, however, the number of patients in each category was too low to assess significance ( Table 2) Figure 2 . By using the cut-offs provided by the manufacturers, a trend was observed in which Abbott IgG yielded positive signals in specimens still negative in Euroimmun IgG and Liaison (RUO) IgG ( Figure 2 ). Rheumatoid factor was detected in five of negative panel specimens (Table 1 ). More detailed results are provided in Tables 1-3. All of the six immunoassays gave reactive results to a varying degree for the negative panel specimens (Table 1) . Particularly the Acro Biotech rapid test and Euroimmun IgA assay reacted in samples retrieved from patients with autoantibodies. As the sensitivity of the Acro Biotech rapid test was lower than the other immunoassays tested, we randomly chose an MNT positive specimen (ID 61), conducted a dilution series of 1:2 for it, and tested the specimen again with the Acro Biotech test. An evident prozone effect was detected, and the originally negative test turned IgG positive at serum dilution 1:4 up until dilution of 1:16. As serological assays for SARS-CoV-2 are now becoming available in the market in abundance [7] , assessment of their analytical performance by using clinical specimens is of critical importance. In this study, we assessed the specificity and sensitivity of six commercial immunoassays for the detection of SARS-CoV-2 antibodies, including two rapid lateral flow tests, in comparison with a neutralisation test. While neutralisation assays are considered to be the gold standard in terms of specificity, they also provide evidence as to development of immunity. Eighty-one of the specimens were retrieved in 2018 and 2019 in Finland, rendering these specimens as ascertained negative for SARS-CoV-2 antibodies, and subsequently verifying the very high specificity of the neutralisation test we used (100 % were negative in MNT). We chose serum dilution 1/40 as the limit of detection for the MNT. RF, which is an autoantibody against the Fc portion of IgG, and a common cause of cross-reactivity in immunoassays [8] , was analysed in the specimens collected in 2018 and 2019. Five out of 39 of these specimens were positive for RF; 4/5 were negative in all SARS-CoV-2 immunoassays, and 1/5 gave a positive reaction in the Acro IgG and IgM test. We conclude that the majority of positive test reactions in the six different immunoassays by using the negative serum panel from 2018-2019 were not due to RF. Of note, we observed a prozone phenomenon [9] by diluting specimen ID 61 for the Acro lateral flow assay. While we did not investigate prozone phenomenon extensively in this study, we do consider it may be an important cause for false negative test results. The prozone phenomenon has been reported for other lateral flow assays previously [10] . Of the automated assays included in this study, and by using the cut-off values set by the manufacturers, the best specificity values were observed with Abbott IgG (95.1%). A previous report from the United States reported a 99.9% specificity [11] . In our study, Liaison IgG (RUO) assay (94.9%) also showed a good specificity. Euroimmun SARS-CoV-2 IgA assay had the best positive agreement (sensitivity) (87.8%), while the positive agreement of the Liaison IgG (RUO) assay was the lowest (43.8%). The CE marked Diasorin Liaison SARS-CoV-2 IgG assay was not available for this evaluation. The automated assays from the three manufacturers were all based on different antigen components (S1, S2, nucleocapsid). This is noteworthy, as antibody responses against each of these antigens may develop with varying kinetics, which remains a subject for further investigation. In addition, the immunoassays may detect nonneutralizing antibodies, not detected by neutralization assays. However, the topic of interest in our study was specifically on comparability of the assays with neutralising antibodies. When interpreting sensitivity values, the time from onset of illness in COVID-19 patients needs to be accounted for. By using the Abbott IgG assay, SARS-CoV-2 IgG seroconversion was previously reported in all patients by the day 17 post onset of illness [11] . Previous reports suggest a median seroconversion time for SARS-CoV-2 from 11 days [12] to 13 days [13] . The present study also suggests a relatively long period required for serological response to take place (Table 2, Figure 1E ). Even though extensive conclusions cannot be made from our data, Liaison IgG (RUO) appears to turn positive at a later point in time from onset of illness in comparison with the other immunoassays evaluated in our study ( Table 2 ). Perkmann et al (14; 2020) have also reported this phenomenon, and it should be investigated more thoroughly whether antibodies against SARS-CoV-2 S1/S2 antigen, in general, are detected in later time point. Of the two rapid lateral flow assays, the Xiamen IgG/IgM showed a good specificity (97.5% / 88.8%) with a modest positive agreement (sensitivity) (71.9% / 81.3%). In line with a previous report [15] , the performance of the Acro Biotech IgG/IgM rapid test appears not to be adequate for The important role of serology for COVID-19 control Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Evaluation of commercial and automated SARS-CoV-2 IgG and IgA ELISAs using coronavirus disease (COVID-19) patient samples Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR Serological and molecular findings during SARS-CoV-2 infection: the first case study in Finland COVID-19 Illness in Native and Immunosuppressed States: A Clinical-Therapeutic Staging Proposal Developing antibody tests for SARS-CoV-2 Rheumatoid factor in acute viral infections: interference with determination of IgM, IgG, and IgA antibodies in an enzyme immunoassay Antigen excess in modern immunoassays: to anticipate on the unexpected False-Negative Serum Cryptococcal Lateral Flow Assay Result Due to the Prozone Phenomenon Performance Characteristics of the Abbott Architect SARS-CoV-2 IgG Assay and Seroprevalence in Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease 2019 Antibody responses to SARS-CoV-2 in patients with COVID-19 Side by side comparison of three fully automated SARS-CoV-2 antibody assays with a focus on specificity Germany) c LIAISON® SARS-CoV-2 IgG (DiaSorin, Saluggia, Italy) d 2019-nCoV IgG/IgM Rapid Test Cassette Microneutralisation assay were carried out according protocol described by Table 1. Negative serum sample panel consisting of samples collected retrospectively during years 2018-2019, prior the SARS-CoV-2 epidemic All results were determined according to manufacturers´ instructions Diasorin) was research use only (RUO) kit. Abbott SARS-CoV-2 IgG chemiluminescent microparticle immunoassay (CMIA) All results were determined according to manufacturers´ instructions *Two separate samples were available from the patients. First number is the identification code for patient We would like to thank Pamela Österlund (Finnish Institute for Health and Welfare, Helsinki, Finland) for providing the virus strain. We would also like to thank (in alphabetical order) Anu 10-1 * Moderate <40 neg neg neg neg 26 Severe <40 neg pos neg neg 24 Severe <40 neg neg neg neg 38 Severe <40 neg pos neg neg 43 Severe <40 neg pos neg neg 32 Severe *Due to limited kit supply, not all specimens tested with MNT could be analysed with these commercial tests. Table 3 . Specificity and sensitivity of the six commercial immunoassays compared with MNT. MNT titer ≥40 was considered as positive. Equivocal results of the commercial assays were regarded as reactive in this analysis. The total number of specimens tested with MNT, and each of the commercial immunoassays, with their respective results are presented in Tables 1-2.