key: cord-0928153-ngpg4yqc
authors: Allen, N.; Brady, M.; Carrion Martin, A. I.; Domegan, L.; Walsh, C.; Houlihan, E.; Kerr, C.; Doherty, L.; King, J.; Doheny, M.; Griffin, D.; Molloy, M.; Dunne, J.; Crowley, V.; Holmes, P.; Keogh, E.; Naughton, S.; O'Rourke, F.; Kelly, M.; Lynagh, Y.; Crowley, B.; De Gascun, C.; Holder, P.; Bergin, C.; Fleming, C.; Ni Riain, U.; Conlon, N.
title: SARS-CoV-2 Antibody Testing in Healthcare Workers: a comparison of the clinical performance of three commercially available antibody assays
date: 2021-05-26
journal: nan
DOI: 10.1101/2021.05.25.21257772
sha: 70931e260d7b338189e58858017142c2f6b34168
doc_id: 928153
cord_uid: ngpg4yqc

SARS-CoV-2 antibodies are an excellent indicator of past COVID-19 infection. As the COVID-19 pandemic progresses, retained sensitivity over time is an important quality in an antibody assay that is to be used for the purpose of population seroprevalence studies. We compared 5788 healthcare worker (HCW) serum samples on two serological assays (Abbott SARS-CoV-2 anti-nucleocapsid IgG and Roche Anti-SARS-CoV-2 anti-nucleocapsid Total Antibody) and a subset of samples (all Abbott assay positive or grayzone, n=485) on Wantai SARS-CoV-2 anti-spike Antibody ELISA. For 367 samples from HCW with previous PCR-confirmed SARS-CoV-2 infection we correlated the timing of infection with assay results. Overall seroprevalence was 4.2% on Abbott, 9.5% on Roche. Of those with previously confirmed infection, 41% (150/367) and 95% (348/367) tested positive on Abbott and Roche respectively. At 21 weeks (150 days) after confirmed infection, positivity on Abbott started to decline. Roche positivity was retained for the entire study period (33 weeks). Factors associated (P[≤] 0.050) with Abbott seronegativity in those with previous PCR-confirmed infection included sex (male OR0.30;95%CI0.15-0.60), symptom severity (OR0.19 severe symptoms;95%CI0.05-0.61), ethnicity (OR0.28 Asian ethnicity;95%CI0.12-0.60) and time since PCR diagnosis (OR2.06 for infection 6 months previously;95%CI1.01-4.30. Wantai detected all previously confirmed infections. In our population, Roche detected antibodies up to at least seven months after natural infection with SARS-CoV-2. This may indicate that Roche is better suited than Abbott to population-based studies. Wantai demonstrated high sensitivity but sample selection was biased. The relationship between serological response and functional immunity to SARS-CoV-2 infection needs to be delineated.

Page 6 of 38 were determined to be reactive and interpreted as antibody positive. The Abbott SARS-CoV-2 IgG 113 grayzone is an additional assay threshold band for potential positivity, suggested by the manufacturer to 114 increase assay sensitivity (Abbott Diagnostics Product Information Letter PI1060-2020) (16), see Table 1  115 for interpretation of the Abbott S/C index within this study. All samples with an Abbott result of positive 116 or grayzone were tested on a third assay in the National Virus Reference Laboratory (NVRL) using the 117

Wantai SARS-CoV-2 Antibody ELISA (referred to here as Wantai), distributed by Fortress Diagnostics. 118

Wantai is an Enzyme-Linked Immunosorbent Assay (ELISA) for qualitative detection of total antibodies 119 (including IgG and IgM) to the spike protein of SARS-CoV-2. 120 121 In terms of assay performance, according to the manufacturer's specifications all three assays perform 124 with high sensitivity and high specificity, see Table 2 . (19) . COVID-19 serology test results were available for 5,788 participating HCWs (comprising 64% of all staff 145 in two Irish tertiary referral hospitals). The majority of participants were female (77%); median age was 146 39 years (IQR 30-49). A small proportion (5%) of participants were over 60 years of age. By role, the 147 highest proportion of participants were nursing staff (36%). Characteristics of participants by serology 148 assay are shown in Appendix 1A and 1B. 149 150

All 5,788 participants were tested using the Abbott SARS-CoV-2 IgG assay and 99.9% (n=5,787) were 152 tested using the Roche Anti-SARS-CoV-2 assay. A considerably lower proportion of participants had a 153 positive antibody on the Abbott assay (4.2%) compared to the proportion that had a positive antibody 154 on the Roche assay (9.5%) ( Table 3) . 155 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) Assay concordance was moderate (k -0.53; 95% CI: 0.48 to 0.57) ( Table 4 ). McNemar's test for 162 difference in proportions indicated a systematic difference in the proportion of positive results between 163 the two assays (p<0.001). Twenty-four participants tested positive on Abbott but negative on Roche. Of 164

the 24, three also tested positive on the Wantai assay, suggesting possible false negative results on 165

Roche for these three participants. Of these three, one was recently diagnosed with COVID-19 (positive 166 by PCR 16 days prior to serology testing). Of the remaining 21, all tested negative on Wantai; none had 167 had prior PCR-confirmed SARS-CoV-2 infection; 11/21 had never been tested and 10/21 had had a 168 negative COVID-19 PCR test at some stage. Of the 21, nine (43%) reported ever having symptoms of 169 COVID-19 (indicating possible undiagnosed infection), eight of whom had mild symptoms (similar to a 170 cold or less) and one of whom had significant symptoms (similar to influenza but not requiring hospital 171 admission). The interval between date of previous symptoms and date of serology testing varied; less 172 than one month (n=1), one month (n=1), two months (n=2), three months (n=1), five months (n=1), six 173 months (n=1), seven months (n=2). Analysis was carried out in order to explore the association between 174 participant characteristics and discordant results between the Abbott and Roche assays, there was no 175 significant association observed (data not presented here). 176 177 178 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) In order to explore whether the numerical value within the grayzone S/C index range could be used to 189 assist interpretation of the grayzone, arbitrary cut-offs (low, medium, high) were applied, and these 190 were compared to the interpreted results on the Roche assay. There was no correlation observed, as a 191 similar proportion of results within each of the arbitrary cut-offs were positive on the Roche assay ( 

Ab ELISA 203

In total, 8.4% (n=485) of participants were tested using both the Roche assay and the Wantai assay. 204

Assay concordance was almost perfect (k -0.93; 95% CI: 0.88 to 0.97) ( Table 6 ). There was no evidence 205 of difference in the proportion of positive results between the two assays (p<0.131). 206 207 

Three hundred and sixty-seven participants were previously diagnosed COVID-19 positive by PCR. The self-reported date of previous positive PCR test was available for 365 (99%) participants. The 220 interval between date of previous positive PCR test and date of serology test ranged from 12 to 231 days 221

(2-33 weeks; 0-7 months). Serology test result by number of weeks since positive PCR test (including the 222 breakdown for those who were symptomatic at the time of PCR testing) is shown in Table 7 and visually 223 represented in Figure 1a , b and c (visual representation excludes participants with grayzone results). We 224 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 26, 2021. saw a decline in antibody positivity on the Abbott assay from week 21 (day 150) onwards. Figure 2  225 shows the percentage positivity by time (months) since PCR test, including 95% confidence intervals (CI), 226

showing a decline in antibody positivity on the Abbott assay in month 4. 227

The most common interval between positive PCR test and serology testing was six months (61%; n=222). 229

There were 210 participants who had a six-month PCR to serology testing interval, and who were 230 symptomatic at the time of their PCR test; among them positivity was 35% (95% CI 29-41) on Abbott 231 (n=73) and it was 93% (95% CI 89-96) for Roche (n=196). proportion were of white Irish background (94% versus 65% in the overall PCR-positive subgroup). Other 240 characteristics did not differ considerably. Analysis was carried out in order to explore the association 241 between participant characteristics and negative serology results; there was no significant association 242 observed (data not presented here). 243

In order to explore whether the Roche quantitative COI or the Abbott S/C index was close to the positive 245 threshold for these 17 participants, arbitrary cut-offs (low, medium and high negative) were applied to 246 the results of both assays. For the Roche assay, three participants had results that were close to the 247 positive result threshold (i.e. in the high negative range: COI 0.6-0.9), six had results in the medium 248 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 26, 2021. ; https://doi.org/10.1101/2021.05.25.21257772 doi: medRxiv preprint Table 7 Detection of SARS-CoV-2 antibodies by serology assay type with respect to time (number of weeks) since positive PCR test 254 indicates 95% confidence interval (CIs). Note: CIs are wide for months 0-4 and month 7 due to a low number of participants at these testing 262

intervals. Figure 2A (Abbott only), 2B (Roche only) 263

The characteristics of participants with previous PCR-confirmed SARS-CoV-2 infection (n=367) and their 265 serology test results by Abbott SARS-CoV-2 are shown in Appendix 3A and 3B. In total, 45% (n=124) (34% 266 including those who had grayzone results in the total number tested) of participants with previous PCR-267 confirmed SARS-CoV-2 infection did not have detectable antibodies on the Abbott assay. Univariable 268 and multivariable analysis were carried to out to explore the association between participant 269 characteristics and negative Abbott test result. To broaden the analysis, multivariable logistic regression was repeated including participants that had 277 grayzone results. The results of this analysis were similar to the results of the initial analysis and are 278 presented in Appendix 4. Separate analysis was carried out to explore the association between 279 participant characteristics and Abbott grayzone results; there was no significant association observed 280 (data not presented). 281

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) In agreement with recently published data (12), a considerably higher proportion of participants 293 (more than double) had detectable antibodies on the Roche assay compared to the Abbott assay. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 26, 2021. ; https://doi.org/10.1101/2021.05.25.21257772 doi: medRxiv preprint There was no obvious correlation between increasing or decreasing Abbott grayzone S/C index and 341 the interpreted results from other assays. In an epidemiological study setting such as this, inclusion 342 of all grayzone results as presumptive positives would lead to significant overestimation of 343 seroprevalence. Studies such as ours could be used to estimate the proportion of grayzone results 344 likely to be positive on other testing platforms, however the use of alternative assays with prolonged 345 sensitivity over time would be superior if the primary purpose is to estimate infection ever. It is 346 notable that while studies have shown good protection against reinfection over a six month period 347

following PCR-confirmed COVID-19 infection (4), to the best of our knowledge no studies have yet 348 compared antibody assays in terms of functional immunity. 349

We found almost perfect agreement between the Roche and Wantai assays. A study comparing 351 eight assays found these two assays to provide the highest sensitivities at 98 and 95 percent 352 respectively (25). However, in our study the assay concordance should be interpreted with caution 353 as selection of participants for additional testing on Wantai ELISA assay was heavily biased; only 354 participants who had a positive or grayzone Abbott SARS-CoV-2 IgG result were selected for testing 355 by Wantai assay. The small degree of discordance between these two assays could be due to a 356 number of reasons. Although these two assays both measure SARS-CoV-2 total antibodies, they are 357 based on different methodology (Roche Elecsys Anti-SARS-CoV2 Total AB is based on 358 chemiluminescence and targets the nucleocapsid, whereas the Fortress Wantai Total AB assay is 359 based on ELISA and targets the spike protein). Furthermore, there is a difference in expected 360 performance of these assays in terms of specificity and sensitivity, according to manufacturer 361

Seropositivity over time (previously PCR-positive participants) 363 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The majority of participants had a six-month interval between confirmed infection and antibody 372 testing in our study, which correlated with the time period between the peak of the first wave of the 373 pandemic in Ireland and the antibody testing in our study. Although pick-up of these was slightly 374 lower on both assays than the overall pickup of previous infections occurring at any time, the Roche 375 still performed significantly better than Abbott using the 6-month timeframe, identifying 93% versus 376 46% of infections that occurred six months prior. 377

The decay in seropositivity on Abbott in our cohort started at 21 weeks (150 days) after confirmed 378 infection, but the small numbers of infection per week prior to this should be noted. This did not 379 change with removal of participants who had no symptoms at the time of infection (Table 7) . A 380 study comparing four different serological assays, including the Abbott and Roche assays, showed a 381 decline in the performance of the Abbott assay after 60 days, whereas antibodies were still detected 382 on the Roche assay after 80 days (12). In contrast, another study of the Abbott assay showed a mean self-reported symptoms were independently associated with higher maximum anti-nucleocapsid IgG 402 levels (13). This is in keeping with our findings on sex, ethnicity and prior self-reported symptoms, 403 however we did not find any statistically significant correlation with age. The reason for this 404 sustained IgG response in those of male sex is not yet clear but may be related to higher viral load, 405 other indices of severity, or other unknown biological factors not included in this study. 406

Participants who had a six-month PCR to serology testing interval were twice as likely to be IgG 407 seronegative when compared to participants who had a five-month testing interval. Those who had 408 a seven-month testing were also more likely to be IgG seronegative when compared to those who 409 had a five-month testing interval, but results were not significant. Our findings are consistent with 410 the findings of other researchers who have demonstrated decline in IgG seropositivity over time, 411

using the Abbott and other IgG antibody assays (7) (26). Further studies may provide better 412 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 26, 2021. ; https://doi.org/10.1101/2021.05.25.21257772 doi: medRxiv preprint understanding of seropositivity and seroreversion over time, relating to specific assays or type of 413 immunoglobulin measured. 414

We did not have enough participants with very recent infection to assess the ability of each assay to 415 pick up early infection, however of the eight infections that occurred within four weeks of antibody 416 testing, 8/9 were correctly identified by the Abbott assay and 6/9 were identified by the Roche 417 assay. Higher numbers would be needed to compare these assays specifically in the early stages of 418

Almost 5% of infections were not identified by either platform. The majority were distant infections 420 (≥6 months ago), and therefore waning immunity may explain the seronegativity. One was a recent . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 26, 2021. ; https://doi.org/10.1101/2021.05.25.21257772 doi: medRxiv preprint Our study has several limitations. Firstly, the study was not designed with the primary objective of 438 comparing the serological assays; two anti-nucleocapsid assays +/-additional testing with an anti-439 spike assay were used to maximise sensitivity in detection of SARS-CoV-2 antibodies. Secondly, 440 information on COVID-19 symptoms and PCR test results were self-reported and thus could be 441 biased. The dates of PCR-confirmed infection (also self-reported) may be inaccurate, furthermore 442 the PCR cycle threshold (Ct) value which would have been a valuable addition to this study was not 443 available. Other variables which would have been valuable to this study but were not available 444 include participant co-morbidities. A small sample size for the 0-4-month PCR to serology test 445 interval prevented meaningful analysis of seropositivity and seronegativity at the early stages post 446

Our study focused on assays that have SARS CoV-2 nucleocapsid as an antigenic target with only a 448 subpopulation assessed using a spike antibody assay. Given emerging evidence of differences in 449 antibody decay related to the antigenic target a more complete assessment would have been 

The Roche assay performed significantly better at picking up those who had ever had a confirmed 457 COVID-19 infection, though both the Roche and Abbott assays were less sensitive than the 458 manufacturers' stated guidelines. Our study findings suggest that, of these two assays directed at 459

anti-nucleocapsid antibodies, Roche is better suited to future population-based serological studies, 460 due to maintained detection of total antibodies up to at least seven months after natural infection 461 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. it is yet to be determined if measuring antibody response to vaccination is meaningful and cost-484 effective, and if so which assays are superior. Further studies are also needed to delineate the 485 relationship between serological response and functional immunity to SARS-CoV-2 infection, both 486 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) 

None of the authors have any conflicts of interest to declare. 503

Ethical approval was obtained from the National Research Ethics Committee for COVID-19 in Ireland 505 (20-NREC-COV-101). 506 507 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 26, 2021. ; https://doi.org/10.1101/2021.05.25.21257772 doi: medRxiv preprint 508 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) .1 *Participants were asked which one describes MOST of their current work ** excludes 27 participants who were not symptomatic at the time of their positive PCR test

Testing for SARS-CoV-2 antibodies

Safety and 511 Immunogenicity of Two RNA-Based Covid-19 Vaccine Candidates

Immunology of COVID-19: 514 Current State of the Science

Prior SARS-CoV-2 infection is associated with protection 516 against symptomatic reinfection

Antibody responses to SARS-CoV-2 in 518 patients of novel coronavirus disease

Antibody response 520 to SARS-CoV-2 infection in humans: A systematic review

Rapid 523 Decay of Anti-SARS-CoV-2 Antibodies in Persons with Mild Covid-19

Change 526 in Antibodies to SARS-CoV-2 Over 60 Days Among Health Care Personnel in Nashville

Humoral Immune Response to SARS-CoV-2 in Iceland

Robust neutralizing 532 antibodies to SARS-CoV-2 infection persist for months

National SARS-CoV-2 Serology Assay Evaluation Group. Performance characteristics of five 535 immunoassays for SARS-CoV-2: a head-to-head benchmark comparison

SARS-CoV-2 IgG Architect -Instructions for Use, FDA

System SARS-COV-2IgG/lgM e-Assay CD-ROM -WW (excluding US) List number 550 6514-03. Information update 07th

Elecsys® Anti-SARS-CoV-2 Immunoassay for the qualitative detection of antibodies (incl. IgG) 552 against SARS-CoV-2

Prevalence of Antibodies to SARS-CoV-2 in Irish Healthcare Workers Phase 1 October

SARS-CoV-2 S1 and 564 N-based serological assays reveal rapid seroconversion and induction of specific antibody 565 response in COVID-19 patients

567 Prevalence of SARS-CoV-2 in Spain (ENE-COVID): a nationwide, population-based 568 seroepidemiological study. The Lancet

Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) in a Large Cohort of Previously 571 Infected Persons

Insights from Patterns of SARS-CoV-2 573

Test Results from a National Clinical Laboratory, United States

Serological 576 reconstruction of COVID-19 epidemics through analysis of antibody kinetics to SARS-CoV-2 577 proteins

580 Quantification of SARS-CoV-2 antibodies with eight commercially available immunoassays

Comparison of the Clinical Performances of the Abbott Alinity IgG

Appendix 1A Characteristics of participants tested, by serology assay 601

Roche Anti-SARS-CoV-2Wantai SARS-CoV-2 Antibody ELISA ^Participants were asked which type of patient contact describes MOST of their current work (excludes five unknowns)