key: cord-0767378-fij6f4y9
authors: Mahajan, S.; Redlich, C. A.; Wisnewski, A. V.; Fazen, L. E.; Rao, L. V.; Kuppusamy, K.; Ko, A. I.; Krumholz, H. M.
title: Performance of Abbott Architect, Ortho Vitros, and Euroimmun Assays in Detecting Prior SARS-CoV-2 Infection
date: 2020-07-30
journal: nan
DOI: 10.1101/2020.07.29.20164343
sha: 035e05e47818f54e81f785d2f594206e8f587bd7
doc_id: 767378
cord_uid: fij6f4y9

Background: Several serological assays have been developed to detect anti-SARS-CoV-2 IgG antibodies, but evidence about their comparative performance is limited. We sought to assess the sensitivity of four anti-SARS-CoV-2 IgG enzyme-linked immunosorbent assays (ELISA) in individuals with evidence of prior SARS-CoV-2 infection. Methods: We obtained sera from 36 individuals with PCR-confirmed SARS-CoV-2 infection between March and May 2020. We evaluated samples collected at around 21 days ({+/-}14 days) after their initial PCR test using 3 commercially available ELISA assays, two anti-spike (Ortho-Clinical Diagnostics Vitros, and Euroimmun) and one anti-nucleocapsid (Abbott Architect), and a Yale-developed anti-spike ELISA test. We determined the sensitivity of the tests and compared their results. The Euroimmun and Yale ELISA had an equivocal and indeterminate category, which were considered as both negative and positive. Results: Among the 36 individuals with SARS-CoV-2 infection, mean age was 43 ({+/-}13) years and 19 (53%) were female. The sensitivities of the tests were not significantly different (Abbott Architect, Ortho Vitros, Euroimmmun, and Yale assays: 86% (95% confidence interval [CI], 71-95), 94% (95% CI, 81-99), 86% (95% CI, 71-95), and 94% (95% CI, 81-99), respectively; p-value=0.464). The sensitivities of the Euroimmun and Yale ELISA tests increased when the equivocal/indeterminate results were considered positive (97% [95% CI, 85-100] and 100% [95% CI, 90-100], respectively), but were not significantly different from other tests (p=0.082). The cross-correlation coefficient ranged from 0.85-0.98 between three anti-spike protein assays (Ortho Vitros, Euroimmun, Yale) and was 0.58-0.71 between the three anti-spike protein assays and the anti-nucleocapsid assay (Abbott). Conclusion: The sensitivities of four anti-SARS-CoV-2 protein assays did not significantly differ, although the sample size was small. Sensitivity also depended on the interpretation of equivocal and indeterminate results. The strongest correlations were present for the three anti-spike proteins assays. These findings suggest that individual test characteristics and the correlation between different tests should be considered when comparing or aggregating data across different populations studies for serologic surveillance of past SARS-CoV-2 infection.

3 Conclusion: The sensitivities of four anti-SARS-CoV-2 protein assays did not significantly differ, although the sample size was small. Sensitivity also depended on the interpretation of equivocal and indeterminate results. The strongest correlations were present for the three antispike proteins assays. These findings suggest that individual test characteristics and the correlation between different tests should be considered when comparing or aggregating data across different populations studies for serologic surveillance of past SARS-CoV-2 infection.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 30, 2020. . https://doi.org/10.1101/2020.07.29.20164343 doi: medRxiv preprint BACKGROUND Serological testing is being increasingly used to evaluate people for antibodies to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Recently, a number of serology tests have been developed and have received Food and Drug Administration (FDA) Emergency Use Authorization (EUA) for the detection of these antibodies. 1 However, there are relatively few evaluations of these tests from a cross-section of individuals with polymerase chain reaction test confirmed SARS-CoV-2 infection.

Accordingly, we sought to determine the sensitivity of three commercially available SARS-CoV-2 IgG assays and an anti-spike enzyme-linked immunosorbent assay (ELISA) test at Yale University, in preparation for conducting a large seroprevalence survey of a representative population sample for the state of Connecticut, USA. We compared the Abbott Architect, the Ortho Vitros and Euroimmun, all of which are approved by the US FDA. The Abbott Architect SARS-CoV-2 IgG assay detects antibodies against the nucleocapsid, whereas the Ortho Vitros anti-SARS-CoV-2 IgG test and Euroimmun SARS-CoV-2 ELISA detect antibodies against the spike protein of the virus. 2 We also compared the results of these antibody tests with each other.

We took advantage of previously collected and de-identified serum specimens from 36 individuals with confirmed SARS-CoV-2 infection between March and May 2020 and the availability of resources to test them with the commercially available assays, which set the sample size. Patients were defined as SARS-CoV-2 positive based on reverse transcription polymerase chain reaction (RT-PCR) testing on nasopharyngeal swab specimens collected.

The serology samples included individuals with variable severity and duration of symptoms, but all samples were collected when individuals were asymptomatic and in convalescence, . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 30, 2020. . https://doi.org/10.1101/2020.07.29.20164343 doi: medRxiv preprint approximately 21 days (±14 days) after their initial RT-PCR test. For non-hospitalized individuals, answers to questions from survey data and telephone interviews were used to identify individuals who self-reported symptoms that they attributed to COVID-19 infection. All individuals provided informed consent and samples were collected in accordance with protocols approved by the Institutional Review Board at Yale University.

Blood was collected in vacutainer tubes, and serum was separated. Serum aliquots were stored at -80°C. Samples had previously been assayed on a laboratory developed test at Yale, an ELISA against the spike antigen, but samples were selected blinded from these results. 

We collected data on the optical density of the Yale anti-spike ELISA and the index values for Abbott Architect, Ortho Vitros, and Euroimmun SARS-CoV-2 IgG assays, and compared the results reported by the four serology assays. The sensitivity of different assays . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 30, 2020. . https://doi.org/10.1101/2020.07.29.20164343 doi: medRxiv preprint was calculated, and Fishers exact test was used to detect for statistical difference. The correlation between the index values of different assays and optical density of Yale anti-spike ELISA was assessed using Spearman's correlation coefficient, with confidence intervals (CI) calculated by z-transformation. Individual scatter plots were created for each two-way test comparison with non-parametric locally weighted regression (Loess) smoothing with 95% CI.

SAS and R (version 3.6.1) was used for all statistical analyses and graphical and tabular preparations. A p-value of <0.05 was considered statistically significant.

Of the 36 individuals with SARS-CoV-2 infection included in the study, mean age 43 (±13) years and 19 were female, 16 were male, and 1 did not have information available on gender.

Sensitivity of the Abbott Architect, Ortho Vitros, Euroimmun, and Yale ELISA SARS-CoV-2 IgG assays is shown in Table 1 , based on the results in 36 individuals with RT-PCR confirmed SARS-CoV-2 infection.

The Abbott Architect IgG assay identified 31 samples as true positives and 5 samples as false negatives, and had a sensitivity of 86% (95% CI, 71% to 95%). The Othro Vitros IgG assay identified 34 samples as true positives and 2 samples as false negatives, yielding a sensitivity of 94% (95% CI, 81% to 99%). Euroimmun IgG assay identified 31 samples as true positives, 1 sample as false negative, and 4 as equivocal. The sensitivity of the assay considering equivocal samples as positives was 97% (95% CI, 85% to 100%), whereas the sensitivity was 86% (95% CI, 71% to 95%) when the equivocal samples were considered as negatives. The Yale antispike ELISA identified 34 samples as true positives and 2 as indeterminate. The sensitivity of . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 30, 2020. . https://doi.org/10.1101/2020.07.29.20164343 doi: medRxiv preprint the assay considering the indeterminate as positives was 100% (95% CI, 90% to 100%), whereas the sensitivity was 94% (95% CI, 91% to 99%) when the indeterminate samples were considered as negatives. Overall, sensitivities of the four anti-SARS-CoV-2 protein assays did not differ significantly whether indeterminate/equivocal samples were considered as positives (p=0.082) or as negatives (p=0.464).

The 

This study assessed the performance characteristics of four anti-SARS-CoV-2 serology tests in individuals with a prior SARS-CoV-2 infection. The sensitivities of the tests were not . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 30, 2020. . https://doi.org/10.1101/2020.07.29.20164343 doi: medRxiv preprint significantly different, and the quantitative scales of all assays were strongly correlated with Spearman coefficients greater than 0.58 and p<0.001 in all cases. While the Ortho Vitros and the Euroimmun assays had a very strong correlation with each other, and good correlation with the Yale anti-spike ELISA, the Abbott Architect assay, the only anti-nucleocapsid test, exhibited lower correlation with the three other tests in the comparison.

The 3 different FDA EUA SARS-CoV-2 IgG assays and the Yale anti-spike ELISA used in this study detect antibodies directed against different immunogenic components of SARS-CoV-2. The spike protein is the main surface glycoprotein of SARS-CoV-2 that is used to attach and enter cells, 6 whereas the nucleocapsid is critical for viral transcription and replication. Initial studies suggested that assays using the SARS-CoV-2 nucleoprotein may have higher sensitivity. 7, 8 However, the data presented here show a qualitatively higher sensitivity with the 3 assays that use the spike glycoprotein (Ortho Vitros, Euroimmun, Yale ELISA), as compared with the Abbott Architect assay, suggesting a need for further comparative testing. Additionally, as expected, the 3 anti-spike assays had a stronger correlation among themselves as compared with the anti-nucleocapsid assay.

Given that the sensitivities of four anti-SARS-CoV-2 protein assays did not significantly differ, it may be reasonable to use any of these assays to assess the prevalence of SARS-CoV-2 antibodies. However, recognizing differences in the correlation between these assays can help facilitate interpretation of prevalence studies using the different assays. For example, given the very strong correlation between Euroimmun and Ortho Vitros assays, it would be reasonable to aggregate seroprevalence studies that use these assays into meta-analysis of seroprevalence or a comparison over time. The weaker correlation of the Abbott Architect with the other serology assays, however, suggest caution should be used when aggregating seroprevalence rates detected by the Abbott Architect assay with those using other serologic assays. Studies focusing on assessment of seroprevalence of COVID-19 in the different populations should consider for the sensitivity of the assay used, and the likely prevalence of . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 30, 2020. . https://doi.org/10.1101/2020.07.29.20164343 doi: medRxiv preprint antibodies in the population, in order to accurately interpret the results. It is also important to know the thresholds for positivity and how tests in the indeterminate range are reported.

Our study has several limitations. Our comparative study focused on the distribution and correlation of anti-SARS-CoV-2 IgG ELISA results among individuals who had confirmed COVID-19, and we were not able to evaluate the specificities of the tests. In addition, our sample size limited our power to detect clinically meaningful differences in sensitivities. Another limitation is that we had limited information on the clinical characteristics of these individuals, which could have affected the nature of antibody response. Nevertheless, this study provides important information on the correlations between 3 commercially available serology tests, as well as the Yale anti-spike ELISA, and also information on the sensitivity of these tests, which can help guide future research on the seroprevalence of IgG antibodies to SARS-CoV-2.

In conclusion, this study shows that the 3 commercial SARS-CoV-2 IgG assays (Abbott . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 30, 2020. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 30, 2020. . https://doi.org/10.1101/2020.07.29.20164343 doi: medRxiv preprint Figure 1 . Correlations between the Yale Anti-Spike ELISA, Abbott Architect, Ortho Vitros, and Euroimmun SARS-CoV-2 IgG Assays. The dashed line represents the cutoffs used.

Abbreviations: CI, confidence interval; ELISA, enzyme-linked immunosorbent assay; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 30, 2020. . https://doi.org/10.1101/2020.07.29.20164343 doi: medRxiv preprint

Performance Characteristics of the Abbott Architect SARS-CoV-2 IgG Assay and Seroprevalence in

Abbott Architect SARS-CoV-2 IgG. Instructions To Use

Ortho-Clinical Diagnostics VITROS Anti-SARS-CoV-2 IgG test. Instructions To Use

Instructions for Use

Structure, Function, and Antigenicity of the SARS-CoV-2 Spike Glycoprotein

Sensitivity in Detection of Antibodies to Nucleocapsid and Spike Proteins of Severe Acute Respiratory Syndrome Coronavirus 2 in Patients With Coronavirus Disease

Serodiagnostics for Severe Acute Respiratory