key: cord-0943606-hb2bpyag authors: Jääskeläinen, A. J.; Kuivanen, S.; Kekäläinen, E.; Ahava, M. J.; Loginov, R.; Kallio-Kokko, H.; Vapalahti, O.; Jarva, H.; Kurkela, S.; Lappalainen, M. title: Performance of six SARS-CoV-2 immunoassays in comparison with microneutralisation date: 2020-05-22 journal: nan DOI: 10.1101/2020.05.18.20101618 sha: 36578e3e9898089389530f4fd8007837aa3c3973 doc_id: 943606 cord_uid: hb2bpyag There is an urgent need for reliable high-throughput serological assays for the management of the ongoing COVID-19 pandemic. Preferably, the performance of serological tests for a novel virus should be determined with clinical specimens against a gold standard, i.e. virus neutralisation. We evaluated specificity and sensitivity of six commercial immunoassays for the detection of SARS-CoV-2 IgG, IgA and IgM antibodies, including four automated assays [Abbott SARS-COV-2 IgG (CE marked), Diasorin Liaison SARS-CoV-2 S1/S2 IgG (research use only), and Euroimmun SARS-CoV-2 IgG and IgA (CE marked)], and two rapid lateral flow (immunocromatographic) tests [Acro Biotech 2019-nCoV IgG/IgM (CE marked) and Xiamen Biotime Biotechnology SARS-CoV-2 IgG/IgM (CE marked)] in comparison with a microneutralisation test (MNT). Two specimen panels from serum samples sent to Helsinki University Hospital Laboratory (HUSLAB) were compiled: the patient panel included sera from PCR confirmed COVID-19 patients, and the negative panel included sera sent for screening of autoimmune diseases and respiratory virus antibodies in 2018 and 2019. The MNT was carried out for all COVID-19 samples (70 serum samples, 62 individuals) and for 53 samples from the negative panel. Forty-one out of 62 COVID-19 patients showed neutralising antibodies with median of 11 days (range 3-51) after onset of symptoms. The specificity and sensitivity values of the commercial tests against MNT, respectively, were as follows: 95.1%/80.5% (Abbott Architect SARS-CoV-2 IgG), 94.9%/43.8% (Diasorin Liaison SARS-CoV-2 IgG), 68.3%/87.8% (Euroimmun SARS-CoV-2 IgA), 86.6%/70.7% (Euroimmun SARS-CoV-2 IgG), 74.4%/56.1% (Acro 2019-nCoV IgG), 69.5%/46.3% (Acro 2019-nCoV IgM), 97.5%/71.9% (Xiamen Biotime SARS-CoV-2 IgG), and 88.8%/81.3% (Xiamen Biotime SARS-CoV-2 IgM). This study shows variable performance values. Laboratories should carefully consider their testing process, such as a two-tier approach, in order to optimize the overall performance of SARS-CoV-2 serodiagnostics. Serosurveys are considered essential for creating timely snapshots for global and regional public health management of the ongoing COVID-19 pandemic (Winter and Hegde, 2020) . Thus, there is an urgent need for the development of high-throughput serological assays, which enable population screening, as well as other epidemiological investigations. Setting up a serological assay for a completely novel pathogen is challenging in many respects. At present, there is inadequate knowledge as to when and what kind of immune response follows SARS-CoV-2 infection (Okba et al. 2020) We are also yet to learn about factors that may disturb reliable serology, such as potential cross reaction from seasonal coronaviruses. The aim of this study was to evaluate the performance of four automated immunoassays for SARS-CoV-2 serology from internationally recognized providers: Abbott, Diasorin, and Euroimmun. In addition, we evaluated two rapid lateral flow assays. The evaluation of Abbott SARS-COV-2 IgG (CE marked), Diasorin Liaison ® SARS-CoV-2 S1/S2 IgG (research use only), and Euroimmun SARS-CoV-2 IgG and IgA (CE marked), and two rapid lateral flow (immunocromatographic) tests of Acro Biotech 2019-nCoV IgG/IgM (CE marked) and Xiamen Biotime Biotechnology SARS-CoV-2 IgG/IgM (CE marked), was conducted against a SARS-CoV-2 microneutralisation test (MNT) by using clinical serum specimens. The patient panel consisted of serum samples from coronavirus 19 disease patients, who had been diagnosed by PCR-based methods from nasopharyngeal samples in our laboratory. For molecular testing, three different methods were used: cobas® SARS-CoV-2 test on the Cobas® 6800 system (Roche Diagnostics, Basel, Switzerland), Amplidiag® COVID-19 test (Mobidiag, Espoo, Finland) and a protocol based on Corman et al (2020) . In total, 70 serum samples from 62 individuals (median age 54 years, range 24-86 years; 28 males, 34 females; Table 2 ) were available for this study. Data were collected and samples treated according to permit HUS/32/2018 (Helsinki University Hospital, Finland). The analysis of SARS-CoV-2 IgG or IgA antibodies were carried out using the Architect Plus i2000sr Analyzer (Abbott, Illinois, USA) and SARS-COV-2 IgG kit (nucleoprotein based antigen; Abbott; CE marked), EUROLabworkstation (Euroimmun, Lübeck, Germany) and SARS-CoV-2 IgG and IgA kits (S1-based antigen; Euroimmun) and Diasorin Liaison ® XL (DiaSorin, Saluggia, Italy) and SARS-CoV-2 S1/S2 IgG kit (S1/S2 based antigen; DiaSorin; research use only) according to manufacturers´ instructions. All samples from the negative panel (N=81) and the patient panel (N=70) were tested with Abbott SARS-CoV-2 IgG, and Euroimmun SARS-CoV-2 IgA and IgG. All samples from the negative panel (N=81) and (due to limited kit supply) 61/70 samples (53/62 individuals) from the patient panel were tested with DiaSorin SARS-CoV-2 S1/S2 IgG. 2019-nCoV IgG/IgM (Acro Biotech, California, USA; CE marked)] and SARS-CoV-2 IgG/IgM (Xiamen Biotime Biotechnology, Fujian, China; CE marked) rapid lateral flow (immunocromatographic) tests were evaluated. Altogether, 53/81 samples from the negative panel and all 70 specimens from the patient panel were tested with Acro Biotech, and 51/81 samples from the negative panel and 61/70 from the patient panel were tested with Xiamen Biotime. MNT was conducted for 53/81 of the negative panel and all of the 70 specimens from the patient panel (Tables 1 and 2 ). Microneutralisation assays were carried out for 39 (39/53) samples positive for autoantibodies, three (3/53) samples from patients with primary EBV infection, four (4/53) samples from patients with acute HCoV OC43 infection, and seven (7/53) samples which were sent for testing of respiratory virus antibodies. MNT was performed in a BSL-3 laboratory as described previously (Haveri et al.2020 ) with modifications. Briefly, Vero E6 cells (5 × 10^4/well) were plated on 96 w/p MEM supplemented with 10% of heat-inactivated FBS, glutamine, penicillin and streptomycin. The serum samples were heat-inactivated at 56 °C for 30 min and 2-fold serially diluted in triplicates starting from 1:40 in MEM supplemented with 2% of heat-inactivated FBS, glutamine, penicillin and streptomycin. Fifty plaque-forming units (PFU) of the SARS-CoV-2/Finland/1/2020 strain, passaged five times in Vero E6 cells, were added to the serum dilutions and incubated for 1 h at 37 °C. The growth medium was removed and the virus-serum mixture was added to the cells and incubated for 4 days at 37 °C with 5% CO2. Neutralising antibody levels were assessed by cytopathic effect (CPE) stained with crystal violet. The neutralisation endpoint titer was determined as the endpoint of the serum that inhibited the SARS-CoV-2 infection in at least 2 out 3 parallel wells. The MNT titer ≥ 40 was considered as positive. For 55 COVID-19 patients out of 62, the date of disease onset was available, and disease severity could be rated (mild, moderate or severe; based on Siddiqi et al. (2020) ) (Table 2, Figure 1 ). In the COVID-19 patients included in this study, the earliest time point for the MNT to become positive All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020 . . https://doi.org/10.1101 /2020 was 3 days from onset of illness (patient ID 50), while the furthest time point for a negative MNT was 16 days from onset (patient ID 5) ( Table 2) . Disease severity did not appear to be reflected in the MNT titers of the patients, however, the number of patients in each category was too low to assess significance (Table 2) . Numeric results of Abbott Architect SARS-CoV-2 IgG, Euroimmun SARS-CoV-2 IgA, Euroimmun SARS-CoV-2 IgG and Diasorin Liaison SARS-CoV-2 IgG were plotted against the MNT titer values (Figure 1 ). The geometric mean of the patient panel specimens exceeded the test cut-off at the following MNT titers: Abbott IgG (test cut-off 1.4 index exceeded with geometric mean 3.50 index at MNT titer 80); Euroimmun IgA (test cut-off 1.1 ratio exceeded with geometric mean 3.79 ratio at MNT titer 80), Euroimmun IgG (test cut-off 1.1 ratio exceeded with geometric mean 1.57 ratio at MNT titer 80), and Liaison IgG (test cut-off 15 AU/ml exceeded with geometric mean 45.1 AU/ml at MNT titer 320) ( Figure 1 ). However, the geometric mean in the Euroimmun IgG assay lingered in close proximity (geometric mean 1.57 ratio) of the cut-off (1.1 ratio) still at MNT titer of 160. Specificity and sensitivity for immunoassays were calculated in comparison with MNT, in which MNT titer ≥ 40 was considered positive (Table 3) . Inconclusive results of the commercial assays were regarded as reactive in the performance calculations. Altogether, 53 samples from the negative panel and all of the 70 samples from the COVID-19 patient panel were tested with MNT (Tables 1-3 Figure 2 . Rheumatoid factor was detected in five of negative panel specimens (Table 1) . More detailed results are provided in Tables 1-3. All of the six immunoassays were reactive to a varying degree of the negative panel specimens (Table 1) . Particularly the Acro Biotech rapid test and Euroimmun IgA assay reacted in samples retrieved from patients with autoantibodies. As the sensitivity of the Acro Biotech rapid test was lower than the other immunoassays tested, we randomly chose an MNT positive specimen (ID 61), conducted a dilution series of 1:2 for it, and tested the specimen again with the Acro Biotech test. An evident prozone effect was detected, and the originally negative test turned IgG positive at serum dilution 1:4 up until dilution of 1:16. As serological assays for SARS-CoV-2 are now becoming available in the market in abundance (Petherick, 2020) , assessment of their analytical performance by using clinical specimens is of critical importance. In this study, we assessed the specificity and sensitivity of six commercial immunoassays for the detection of SARS-CoV-2 antibodies, including two rapid lateral flow tests, in comparison with a neutralisation test. While neutralisation assays are considered to be the gold standard in terms of specificity, they also provide evidence as to development of immunity. Eighty-one of the specimens were retrieved in 2018 and 2019 in Finland, rendering these specimens as ascertained negative for SARS-CoV-2 antibodies, and subsequently verifying the very high specificity of the neutralisation test we used (100 % were negative in MNT). We chose serum dilution 1/40 as the limit of detection for the MNT. Failure to detect very low antibody concentrations in this setup is possible. However, four of the 62 PCR-positive individuals showed neutralising antibodies without reactivity in any of the IgG tests used, suggesting a reasonable level of sensitivity in our neutralisation assay. 1 0 RF, which is an autoantibody against the Fc portion of IgG, and a common cause of cross-reactivity in immunoassays (Salonen et al. 1980) , was analysed in the specimens collected in 2018 and 2019. Five out of 39 of these specimens were positive for RF; 4/5 were negative in all SARS-CoV-2 immunoassays, and 1/5 gave a positive reaction in the Acro IgG and IgM test. We conclude that the majority of positive test reactions in the six different immunoassays by using the negative serum panel from 2018-2019 were not due to RF. Of note, we observed a prozone phenomenon (Jacobs et Of the automated assays included in this study, and by using the cut-off values set by the manufacturers, the best specificity values were observed with Abbott IgG (95.1%), which is in line with a previous report from the United States reporting a 99.9% specificity (Bryan et al. 2020 ). In our study, Liaison IgG assay (94.9%) also showed a good specificity. Euroimmun SARS-CoV-2 IgA assay had the best sensitivity (87.8%), while the sensitivity of the Liaison IgG assay was the lowest (43.8%). When interpreting sensitivity values, the time from onset of illness in COVID-19 patients needs to be accounted for. By using the Abbott IgG assay, SARS-CoV-2 IgG seroconversion was previously Of the two rapid lateral flow assays, the Xiamen IgG/IgM showed a good specificity (97.5% / 88.8%) with a modest sensitivity (71.9% / 81.3%). In line with a previous report (Lassaunière et al. 2020) , the performance of the Acro Biotech IgG/IgM rapid test appears not to be adequate for clinical use, with specificity of 74.4% / 69.5% and sensitivity of 56.1% / 46.3%. The currently very low seroprevalence of SARS-CoV-2 in most regions globally render low positive predictive values in the serological testing of individual patients. This can be somewhat improved by good targeting of groups tested. The analytical test performance can be optimised by placing several consecutive assays, with varying antigenic features, in the test workflow, ideally emphasizing sensitivity in the screening and specificity in the second-line testing. The very variable performance values observed in this study highlights the need for laboratories to carefully consider their testing process in order to optimize the overall performance of SARS-CoV-2 serodiagnostics. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. T a b l e 1 . N e g a t i v e s e r u m s a m p l e p a n e l c o n s i s t i n g o f s a m p l e s c o l l e c t e d r e t r o s p e c t i v e l y d u r i n g y e a r s 2 0 1 8 -2 0 1 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. ) n e g / n e g n e g / n e g < 4 0 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020. . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020 . . https://doi.org/10.1101 /2020 e S A R S -C o V -2 I g G / I g M R a p i d T e s t ( X i a m e n B i o t i m e , F u j i a n , C h i n a ) f M i c r o n e u t r a l i s a t i o n a s s a y w e r e c a r r i e d o u t a c c o r d i n g p r o t o c o l d e s c r i b e d b y H a v e r i e t a l . ( 2 0 2 0 ) . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020 . . https://doi.org/10.1101 /2020 (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020. . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. T a b l e 3 . S p e c i f i c i t y a n d s e n s i t i v i t y o f t h e i m m u n o a s s a y s t e s t e d a s c o m p a r e d t o r e s u l t s f r o m m i c r o n e u t r a l i s a t i o n a s s a All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020. . https://doi.org/10.1101/2020.05.18.20101618 doi: medRxiv preprint The important role of serology for COVID-19 control Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR Serological and molecular findings during SARS-CoV-2 infection: the first case study in Finland COVID-19 Illness in Native and Immunosuppressed States: A Clinical-Therapeutic Staging Proposal Developing antibody tests for SARS-CoV-2 Rheumatoid factor in acute viral infections: interference with determination of IgM, IgG, and IgA antibodies in an enzyme immunoassay We would like to thank Pamela Österlund (Finnish Institute for Health and Welfare, Helsinki,