key: cord-0802364-3empldlo authors: Plebani, M.; Padoan, A.; Sciacovelli, L.; Bonfante, F.; Pagliari, M.; Bozzato, D.; Cosma, C.; Bortolami, A.; Negrini, D.; Zuin, S. title: Analytical and clinical performances of five immunoassays for the detection of SARS-CoV-2 antibodies in comparison with neutralization activity date: 2020-08-04 journal: nan DOI: 10.1101/2020.08.01.20166546 sha: 2c7da47bcee8bf3acfd156f00d8ac37af70f5a63 doc_id: 802364 cord_uid: 3empldlo Background. Reliable high-throughput serological assays for SARS-CoV-2 antibodies (Abs) are urgently needed for the effective containment of the COVID-19 pandemic, as it is of crucial importance to understand the strength and duration of immunity after infection, and to make informed decisions concerning the activation or discontinuation of physical distancing restrictions. Methods. In 184 serum samples from 130 COVID-19 patients and 54 SARS-CoV-2 negative subjects, the analytical and clinical performances of four commercially available chemiluminescent assays (Abbott SARS-Cov-2 IgG, Roche Elecsys anti-SARS-CoV-2, Ortho SARS-CoV-2 total and IgG) and one enzyme-linked immunosorbent assay (Diesse ENZY-WELL SARS-CoV-2 IgG) were evaluated and compared with the neutralization activity achieved using the plaque reduction neutralization test (PRNT). Findings. Precision results ranged from 0.9% to 11.8% for all assays. Elecsys anti-SARS-CoV-2 demonstrated linearity of results at concentrations within the cut-off value. Overall, sensitivity ranged from 78.5 to 87.8%, and specificity, from 97.6 to 100%. On limiting the analysis to samples collected 12 days after onset of symptoms, the sensitivity of all assays increased, the highest value (95.2%) being obtained with VITRO Anti-SARS-CoV-2 Total and Architect SARS-CoV-2 IgG. The strongest PRNT50 correlation with antibody levels was obtained with ENZY-Well SARS-CoV-2 IgG (rho = 0.541, p < 0.001). Interpretation. The results confirmed that all immunoassays had an excellent specificity, whereas sensitivity varied across immunoassays, depending strongly on the time interval between symptoms onset and sample collection. Further studies should be conducted to achieve a stronger correlation between antibody measurement and PRNT50 in order to obtain useful information for providing effective passive antibody therapy, and developing a vaccine against the SARS-CoV-2 virus. The continuing spread of coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has prompted concern worldwide, leading the World Health Organization (WHO) to declare COVID-19 a pandemic on 11 March 2020 1 Analytical pitfalls in both the pre-and analytical steps have been described 2 and negative molecular test results have been reported in the later stages of infection, thus being misleading from a clinical viewpoint. Therefore, rRT-PCR precludes the identification of individuals who have been infected, but have had only minor, or no, symptoms and therefore have not sought medical attention. A wide range of immunoassays to detect SARS-CoV-2 antibodies (Ab) have been developed to complement rRT-PCR, with different antigen targets and formats [3] [4] [5] . Although not effective for making an early diagnosis, serological assays for SARS-CoV-2 play an important role in diagnosing COVID-19 disease in individuals who present late, in understanding the virus epidemiology in the general population, and in identifying the disease prevalence in categories at higher risk of infection (e.g. healthcare workers). In addition, they should be used to ascertain the efficacy of containment measures both locally and globally, to screen convalescent sera for therapeutic and prophylactic purposes, and to improve knowledge of the immune response to the novel virus as the degree and duration of the response of specific antibodies is as yet poorly understood 6, 7 . Like infections from other pathogens, SARS-CoV-2 infection elicits development of IgM and IgG specific Ab which are the most available antibodies for assessing response, while little is known about IgA response in the blood. The aim of this paper is to evaluate the performance characteristics and diagnostic specificity, sensitivity of four chemiluminescent assays (CLIA) and one enzyme-linked immunosorbent assay 122 hospitalized classified with moderate or severe disease, following WHO interim guidance 8 ) and 54 SARS-CoV-2 negative subjects (33 healthcare workers, 21 autoimmune patients, 8 pregnant women) were included in the study ( Moreover, Liaison SARS-CoV-2 S1/S2 IgG (Diasorin, Sallugia-VC, Italy), ENZY-Well SARS-CoV-2 IgA and IgM were evaluated for the correlation with the neutralization results. Precision estimation was performed on CLIA assays using two human serum sample pools with different values, by means of quintuplicate measurements of same pool aliquots, performed for a total of four consecutive days. Nested analysis of variance was used to estimate precision, following the CLSI EP15-A3 protocol 9 The results for precision were compared to those claimed by the manufacturer when available, using the procedure recommended by EP15-A3. Repeatability and within-laboratory precision were in accordance with the repeatability and intermediate precision conditions specified in the international vocabulary of metrology (VIM, JCGM 100:2012) for precision estimation within a four-day period. SARS-CoV-2 IgG. The pools were serially diluted with the corresponding low-level serum pools (0.174 S/CO ratio for Elecsys Anti-SARS-CoV-2, 0.01 S/CO for VITROS Anti-SARS-CoV-2 IgG, 0.2 S/CO for VITROS Anti-SARS-CoV-2 total, 0.04 S/CO ratio for Architect SARS-CoV-2 IgG). All measurements were performed in triplicate. For a subgroup of 52 samples from SARS-CoV-2 positive subjects, the Liaison SARS-CoV-2 S1/S2 IgG (Diasorin, Sallugia, VC, Italy) 5 For evaluation of precision, an in-house developed R (R Foundation for Statistical Computing, Vienna, Austria) script for implementing the CLSI EP15-A3 protocol was used for ANOVA and for calculating the upper verification limit 9 module was used to estimate sensitivity, specificity, and positive and negative predictive values. Results for precision of all the CLIA assays are reported in Table 3 . The ANOVA approach allowed us to estimate repeatability and intermediate precision separately. Only the Architect SARS-CoV-2 IgG insert reported data on precision, claimed at levels of 0.04 and 3.53 S/CO ratio. For this immunoassay, intermediate precision performances statistically deviated from the manufacturer's claims at both levels. All the immunoassays had acceptable analytical imprecision (CV%). Linearity results for all the CLIA studies are summarized in Figure 1 . All tested mixes of sample pools covered a wide range of values and included the manufacturers' cut-offs. All immunoassays, except for Elecsys Anti-SARS-CoV-2, deviated from linearity, the coefficients of the second-order polynomial fit attaining high statistical significance. Sensitivities, specificities, and positive and negative likelihood ratios were estimated using the manufacturers' cut-offs, while receiver operating characteristic (ROC) curves were used to evaluate overall performance(s). Elecsys Anti-SARS-CoV-2 immunoassay results were available for All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. . 172/184 (93.4%) serum samples. Table 4 summarizes estimated clinical performances for all CLIA and the ELISA immunoassays considering the total time frame of 93 days and limiting the analyses to sera collected 12 days after the onset of symptoms. One hundred fifty-eight samples were included and evaluated in this restricted subgroup, while only 146 results were available for Elecsys Anti-SARS-CoV-2. Table 4 shows data on positive and negative likelihood ratios, allowing an easy estimation of positive (PPV) and negative (NPV) predictive values given disease prevalence. Considering two different scenarios of disease prevalence settings: (a) 4%, as found in a Veneto Region (Italy) survey (data not shown) 11 ; (b) 10%, as described in a survey conducted in Geneva 12 , PPV and NPV were then estimated, using VITROS Anti-SARS-CoV-2 Total and Architect SARS- Since the results of serum samples and the corresponding immunoassays' cut-offs were used to derive either positive or negative test results, pairwise comparisons of all tests by Cohen's kappa and overall agreements (in percentages) were calculated (Table 5) . The signal-to-cut-off (S/CO) ratios of the examined assays, including Liaison SARS-CoV-2 S1/S2 (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. . In this retrospective study, the analytical and clinical performances of four commercially available CLIA assays and one ELISA assay (Table 2 ) have been evaluated and compared with neutralization activity using the plaque reduction neutralization test (PRNT). The neutralization activity was evaluated also with respect Liaison SARS-CoV-2 S1/S2 IgG and ENZY-WELL SARS-CoV-2 IgA and IgM. Before conducting the study, precision at two concentration levels and linearity were assessed for CLIA by using a standardized protocol according to the CLSI EP15-A3 and CLSI EP06-A (Table 3 and Fig. 1 ) 9, 10 The results obtained demonstrated that both repeatability and intermediate precisions were comparable with other immunoassays performances for the highest concentration levels 3, 5 , whilst for the lowest levels, less satisfactory results were obtained All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. . To provide insight on neutralization activity compared with immunoassays results, PRNT assay was performed on 52 samples from SARS-CoV-2 positive subjects. With the exception of All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. . and an in-house developed IgG ELISA with recombinant RBD of the spike protein as coated antigen 21 . In addition, we found a highly significant correlation between PRNT and IgM Elisa results, thus confirming the data reported by Perera et al. 21 . Another report, which compare IgG or total antibodies measurement of three ELISA, two CLIA and two lateral flow tests, in a total of 100 SARS-CoV-2 convalescent plasma donors, found a good correlation (rho > 0.700) between ELISA (Euroimmun IgG and Wantai Total antibodies) and neutralization titer 22 . In asymptomatic/paucisymptomatics, PRNT 50 titer values were lower than in moderate/severe SARS-CoV-2 patients. Furthermore, a significant negative correlation was found between the PRNT 50 titer and the time interval from symptom onset (Figure 3 ). The present paper has limitations: first, neutralizing antibodies were tested in a limited number of samples the procedure being very complex; second, COVID-19 positive patients were selected retrospectively on the basis of available leftover samples; therefore NPV and PPV could be overestimated. Finally, the relationship between IgM antibodies and neutralizing activity should be studied further in a larger series of patients. In conclusion, although the performances of SARS-CoV-2 antibody immunoassays are of analytical and clinical value, they could be enhanced by considering the test purposes, emphasizing sensitivity in the screening and specificity in the second-line testing. In addition, a further search should be made for a better dynamic range and a stronger correlation with respect to antibody neutralization activity, in order to, above all, obtain information needed for effective passive antibody therapy and vaccine development against SARS-CoV-2 virus. We thank Daniela Rinaldi (medical laboratory scientists) for their valuable technical support. We acknowledge Abbott Laboratories, Diesse Diagnostica Senese, Diasorin, Ortho Clinical Diagnostics, Roche Diagnostic for kindly supplying reagents without any influence in study design and data analysis. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. . (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted August 4, 2020. . Timeline of WHO's response to COVID-19 Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19) Analytical performances of a chemiluminescence immunoassay for SARS-CoV-2 IgM/IgG and antibody kinetics IgA-Ab response to spike glycoprotein of SARS-CoV-2 in patients with COVID-19: A longitudinal study Diagnostic performances and thresholds: the key to harmonization in serological SARS-CoV-2 assays? Antibody responses to SARS-CoV-2 in patients with COVID-19 Serology assays to manage COVID-19 Clinical Management of COVID-19, Interim guidance User Verification of Precision and Estimation of Bias; Approved Guideline-Third Edition. CLSI EP15-A3 Interpreting Diagnostic Tests for SARS-CoV-2 Clinical evaluation of serological IgG antibody response on the Abbott Architect for established SARS-CoV-2 infection Clinical performance of different SARS-CoV-2 IgG antibody tests Shortened title: SARS-CoV-2 IgG antibody test performance