key: cord-0997242-sl6kgo0s
authors: Harrington, David; Azim, Tahira; Rosmarin, Caryn; Cutino-Moguel, Teresa; Hopkins, Mark
title: Evaluation of two commercial anti-SARS-CoV-2 antibody assays in an immunocompetent and immunocompromised inpatient population with COVID-19
date: 2021-06-05
journal: Diagn Microbiol Infect Dis
DOI: 10.1016/j.diagmicrobio.2021.115449
sha: 421f972ed50dd433faa49a8d7601ba765d8bb8f5
doc_id: 997242
cord_uid: sl6kgo0s

INTRODUCTION: During the COVID-19 pandemic, widespread introduction of SARS-CoV-2 antibody testing was introduced without a full understanding of the assays performance or the antibody kinetics following infection with SARS-CoV-2. METHODS: We performed an evaluation of two anti-SARS-CoV-2 antibody assays with a more detailed look into the effect of immune status on antibody sensitivity. RESULTS: Both assays demonstrated a 100% specificity. The overall sensitivity of the Roche was 92.1% at ≥14 days and 94.8% at ≥21 days, and the overall sensitivity of the Abbott was 94.4% at ≥14 days and 98.2% at ≥21 days. 7/41 (17%) of patients included in this cohort were immunocompromised. Seroconversion was seen less commonly in the immunocompromised (4/7 [57.1%] seroconverted) and after excluding these patients 100% sensitivity was seen in both assays at ≥21 days. DISCUSSION: Performance of both assays in the immunocompetent appeared excellent after 21 days post symptom onset. Both assays are highly specific.

As a part the COVID-19 pandemic response there has been an unprecedented introduction of serological testing. The assays predominantly used in the UK are the Roche Elecsys Anti-SARS-CoV-2 antibody assay and the Abbott SARS-CoV-2 IgG assay. The Roche Elecsys Anti-SARS-CoV-2 combined IgM-IgG assay is a modified double sandwich electrochemiluminescence immunoassay (ECLIA) which detects anti-SARS-CoV-2 IgM and IgG targeted against the SARS-CoV-2 virus nucleocapsid (N). The Abbott SARS-CoV-2 IgG assay is a chemiluminescent microparticle immunoassay (CMIA) which detects anti-SARS-CoV-2 IgG targeted against the SARS-CoV-2 virus nucleocapsid (N).

A rapid evaluation of the Roche assay performed between the 5 th and 7 th May 2020 by Public Health England (PHE) using days from symptom onset rather than days from PCR confirmation used samples from 93 SARS-CoV-2 convalescent patients and 472 negative samples and found a specificity of 100%, and a sensitivity of 83.9% 1 . A European diagnostic lab evaluation of similar size found reported a sensitivity of 83.5% for this assay 2 . A more in depth PHE evaluation of the Roche assay with a longer period of follow up reported a 97.2% sensitivity at 20 days using a set of 536 samples 3 . Antibody responses were sustained up to 73 days post symptom onset and up to 82 days post a positive PCR result 3 . An evaluation from the US reported 100% sensitivity after 18 days post symptom onset 4 .

A rapid evaluation of the Abbott assay by PHE using 122 samples from 31 patients reported lower sensitivity of 92.7% ≥14 days post symptom onset and 93.5% ≥21 days post symptom onset, with a lower specificity of 93.9% 5 . In the larger PHE evaluation of 536 positive samples and 994 pre-pandemic samples, a sensitivity of 92.7% was reported at ≥20days post symptom onset, and specificity of 99.9% reported for the Abbott assay 3 . An evaluation from a US diagnostic laboratory reported 100% sensitivity after day 17 post symptom onset 6 .

Both the Roche and the Abbott assays failed to meet UK Medicines and Healthcare products Regulatory Agency (MHRA) Target Product Profile (TPP) for 'enzyme immunoassays' for SARS-CoV-2, which states the assays should have a sensitivity greater than 98% with 95% confidence intervals of 96-100% on specimens collected >20 days when tested on a group of at least 200 positive cases 7 . With 'optimisation of assay thresholds' the Roche assay met the MHRA standard for sensitivity, although the Abbott did not.

Conversely it has been reported that up to 8.5% of those with confirmed SARS-CoV-2 infection do not seroconvert at all, and that this is more common in those with mild or asymptomatic infection 8 . It is now also reported that IgG responses to SARS-CoV-2 can wane quickly and seroreversion can be seen 9-11 .

Published data on the antibody response in the immunocompromised are sparse and largely confined to individual case reports and case series and one small study of immune responses in renal transplant patients 12-20 . Data on performance of these assays in severe vs non-severe groups is limited.

Here we present the results of an evaluation exercise of these two assays including a more detailed look at differences in sensitivity, time to seroconversion, and antibody waning in immunocompetent and immunocompromised groups.

For the uncertainty calculation an in-house internal quality control (IQC) was prepared using a patient sample and serially diluted. For the specificity calculation 50 pre-pandemic samples collected between July to September 2018 from 50 separate patients were retrieved as 'negative' samples. For the sensitivity calculations a larger sample set was identified by cross-matching a list of current inpatients in our hospital trust who were admitted 7 to 14 days previously against a list of all confirmed COVID-19 patients that have been seen by our trust during the pandemic (all patients testing positive for SARS-CoV-2 RNA using a PCR test). Surplus serum samples were retrieved from the virology archive for retrospective SARS-CoV-2 serological testing, and prospective SARS-CoV-2 antibody testing was also undertaken on surplus serum obtained from Blood Sciences laboratory once routine biochemistry testing was completed until each patient was discharged or the period of sample gathering ended. Sample collection and storage occurred between March and May 2020. Clinical information on age, ethnicity, presence of absence of immunocompromise, and ITU admission was collected.

Patient serum samples were tested using the Roche Elecsys Anti-SARS-CoV-2 antibody assay on a Cobas e801 analyser and using the Abbott SARS-CoV-2 IgG assay on a Abbott Architect analyser. For the Roche Elecsys Anti-SARS-CoV-2 combined IgM-IgG assay Cut-off Index (COI)  1.0 is reported as reactive, and COI <1.0 is reported as non-reactive. For the Abbott SARS-CoV-2 IgG assay, COI  1.4 is reported as reactive, and COI <1.4 is reported as nonreactive.

Measurement of anti-SARS-CoV-2 antibody was performed following the manufacturer's instructions with both assays. Measured COIs were measured and documented. Information on, date of onset of symptoms, and immune status were collected.

Assay Verification Local verification of both assays was undertaken prior to implementation. This included external quality assessment panels, inter-laboratory sample exchange and replicate testing of an internal quality control. Observed maximal coefficient of variation above the assay cut-off was 2.61% using Roche and 2.32% using Abbott.

A panel of 50 pre-pandemic serum samples collected July -September 2018 were tested as negative controls. Negative results were recorded for 50/50 (100%) samples using both the Roche and Abbott assays, suggesting both assays had a 100% specificity.

Cross-matching of confirmed Covid-19 cases with a list of all admissions admitted 7-14 days previously identified 79 patients. 23 sera with insufficient sample volume for testing (less than 0.2ml) were excluded. 24 patients (all samples) were excluded due to insufficient time points to assess (minimum 3 time points). A further 14 patients were excluded with unclear symptom onset date.

A total sample set of 388 samples from 41 patients was available for testing. These samples were taken between March and June 2020.Sample collection date post-symptom onset ranged 1 to 120 days. The COIs measured on testing each sample are represented in figure  1 . 7 patients were immunocompromised; 1 on rituximab and long-term steroids for rheumatoid arthritis (RA), 1 on rituximab for lymphoma, 1 new diagnosis of leukaemia, 1 renal transplant on long term prednisolone, azathioprine and ciclosporin, 1 uncontrolled HIV, 1 on long term steroids and hydroxychloroquine for systemic lupus erythematosus (SLE), 1 on long term mesalazine for Crohns disease.

38/41 (92.7%) patients developed detectable anti-SARS-CoV-2 antibody on both assays evaluated. 3/41 (7.3%) had no detectable antibody response with periods of follow up for these 3 patients being 18, 20, and 27 days post symptom onset respectively. Subsequent serum samples collected from these patients after our data collection also had negative anti-SARS-CoV-2 IgM/IgG results. These samples were collected at 185, 81, and 50 days post symptom onset respectively. No additional antibody testing was undertaken in the interim time periods. The severity of illness in the 3 patients who did not seroconvert was moderate, critical, and moderate respectively.

Only 7 patients had samples available with enough frequency from early illness (at least 1 serum every 72 hours until seroconversion) to detect the approximate time of seroconversion post symptom onset by measuring a negative and then a positive antibody on serial testing. Using the Roche assay seroconversion was seen by days 6, 8, 10, 11, 12, 14, and 19 respectively, and using the Abbot assay seroconversion was seen by days 6, 8, 10, 12, 12, 14, and 19 days respectively in these 7 patients. Mean time to seroconversion was 11 days for both assays assessed. Patients were observed to seroconvert up to 36 days post symptom onset. Detectable antibody was observed as early as 4 days post symptom onset with the Roche, and 2 days post symptom onset with the Abbott. In these patients, antibody was detected on the initial serum sample, and so it was not clear at what time point seroconversion actually occurred. 7/41 (17%) of patients in our cohort were immunocompromised. Samples taken from immunocompromised patients had numerically lower rates of positivity at each time point. Only 4/7 (57.1%) of immunocompromised patients seroconverted and only 2 of these had seroconverted by days 21-25 post symptom onset. Seroconversion by time point was similar for both assays tested. A patient with a new diagnosis of leukaemia and 2 patients on rituximab did not demonstrate any antibody response in our cohort. Seroconversion did occurr in one patient with uncontrolled HIV, a patient on mesalazine for Crohns, a patient on long term prednisolone and azathioprine for SLE and a renal transplant patient. When analysing samples taken from immunocompromised patients only, observed sensitivity at 14 days was 72.6% for the Roche and 85.7% for the Abbott. Sensitivity at 21 days was 81% for the Roche and 95.2% for the Abbott (Table 2) .

Since data collection for this project finished, Abbott introduced the option of a 'grayzone' in reporting of results of the Abbott anti-SARS-CoV-2 IgG assay on both ARCHITECT™ and Alinity systems. Laboratories have the option of reporting COIs of 0.5 -<1.4 as a grayzone or of just continuing with reactive/non-reactive result reporting. Reactive results are still reported for COI≥1.4 and only reactive results have been included in these sensitivity analyses.

Abbott sensitivity Antibody waning and seroreversion 1 of 41 patients (2.4%) subsequently lost their detectable antibody response and reverted to seronegativity. This patient was immunocompromised on ciclosporin, prednisolone and azathioprine following a renal transplant. Serial samples from this patient had undetectable anti-SARS-Co-V-2 antibodies by the Roche assay, although a waning antibody response was detected using the Abbott assay. With the latter, weak reactivity was seen initially (peak COI 1. 19) , and seroreversion occurred by day 43 post symptom onset. Interestingly a late sample was sent at day 86 and antibody was again detectable at a higher COI of 5.23 (see patient 7 in figure 2 ). No boost in the antibody response was seen using the Abbott assay. The patient had been discharged in the interim and it is not known if there were any further exposure events which may have boosted antibody occurred. A sample mix-up cannot be excluded.

Evaluation of SARS-CoV-2 IgG antibody response in PCR positive patients: Comparison of nine tests in relation to clinical data

Evaluation of sensitivity and specificity of four commercially available SARS-CoV-2 antibody immunoassays

Differences in Performance Characteristics Among Four High-Throughput Assays for the Detection of Antibodies Against SARS-CoV-2 Using a Common Set of Patient Samples

Performance Characteristics of the Abbott Architect SARS-CoV-2 IgG Assay and Seroprevalence in

Target Product Profile -Antibody tests to help determine if people have immunity to SARS-CoV-2

Dynamics of IgG seroconversion and pathophysiology of COVID-19 infections. Accessed 8/8/2020, 9

Rapid Decay of Anti-SARS-CoV-2 Antibodies in Persons with Mild Covid-19

Disappearance of antibodies to SARS-CoV-2 in a COVID-19 patient after recovery

Longitudinal evaluation and decline of antibody responses in SARS-CoV-2 infection. Accessed 8/8/20, 12

Is serological response to SARS-CoV-2 preserved in MS patients on ocrelizumab treatment?

Online ahead of print

Negative SARS-CoV-2 antibody testing following COVID-19 infection in two MS patients treated with ocrelizumab

Immune monitoring facilitates the clinical decision in multifocal COVID-19 of a pancreas-kidney transplant patient

Control os SARS-CoV-2 infection in rituximab-treated neuroimmunological patients

Failed antibody response in a renal transplant recipient with SARS-CoV-2 infected

Evidence of potent humeral immune activity in COVID19-infected kidney transplant recipients

COVID-19 in 7 multiple sclerosis patients in treatment with anti-CD20 therapies

Influence of immunosuppression on seroconversion against SARS-CoV-2 in two kidney transplant recipients

SARS-CoV-2 IgG for use with Architect

Elecsys Anti-SARS-CoV-2 Immunoassay for the qualitative detection of antibodies against SARS-CoV-2. Product information

Comparison of the diagnostic sensitivity of SARS-CoV-2 nucleoprotein and glycoprotein-based antibody tests

Covid-19: Two antibody tests are "highly specific" but vary in sensitivity, evaluations find

Antibody waning was defined as a progressive sustained average drop in COIs over serial time points. Waning in antibody response was seen in 7 patients -6 using the Roche assay and 4 using the Abbott assay (figure 2). All were immunocompetent except the renal transplant patient (patient 7 in figure 2) who demonstrated seroreversion. 

In this evaluation both the Roche anti-SARS-2 antibody assay and the Abbott anti-SARS-2 IgG assay demonstrated 100% specificity in keeping with high specificities observed in other evaluations 1, 3, 5, 21, 22 . Testing of replicate samples suggested reproducibility was very good and the performance of these assays in our lab was consistent.Early central evaluations, and some local evaluations have used small sample sizes which may have affected sensitivity calculations 1, 5, 23 . The original rapid PHE evaluations which led to the NHS Improvement (NHSi) recommendations for use of these assays drew criticism due to low sample size, and insufficient information about the patients from whom the samples were taken 24 . Studies into assay performance in the immunocompromised have not been published and data on the antibody response in immunocompromised populations are sparse.The group of patients from which these samples were gathered were a typical inpatient population. No staff samples or outpatient samples were included. Male to female ratio, and average age and severity were typical for an inpatient population around peak. Testing a large sample size over time points ranging from 1-120 days post symptom onset we found that 7.3% of patients remained antibody negative which is similar to a previous report 8 .The sensitivity of both assays when analysed by time point sensitivity remained low until day 21 post symptom onset consistent with PHE data 3 . All the immunocompetent patients in our cohort that seroconverted did so on both assays. It was observed that seroconversion occurred less frequently in immunocompromised patients and when samples from immunocompromised patients were excluded, sensitivity of both assays appeared excellent. Seroreversion was seen in 1 immunocompromised patient. There is little data on antibody kinetics in the immunocompromised with only a handful of published case reports. There were two patients in our cohort on rituximab and neither developed detectable antibody. Lack of seroconversion in rituximab and other anti-CD20 agents has been reported elsewhere 13, 14, 25 . A study into the immune response in 18 renal transplant patients, which is the largest cohort of immunocompromised patients with confirmed SARS-CoV-2 infection found that 16/18 developed a detectable antibody response 18 . We had one renal transplant patient in our cohort who did seroconvert, although with very weak COIs measured and a subsequent reversion to seronegativity on day 43 post symptom onset. Overall sensitivity for samples taken from immunocompromised patients using the Roche was low, questioning its utility in this population. It is difficult to draw any definite conclusions about performance in the immunocompromised from this data given the low numbers of patients and samples. In the immunocompromised group, the Abbott assay resulted in numerically higher sensitivities than the Roche, reaching the MHRA sensitivity cut-off for use at ≥21 days.Our dataset was limited by the samples that were available to use. As we were not collecting samples specifically for this evaluation we were reliant on samples sent to the laboratory for routine clinical reasons. We were further limited by the short period of time serum samples were stored. This may have resulted in some skew to our results of sensitivity using the larger set of samples as a whole as multiple samples were included from each patient with varied and inconsistent time points available for analysis across patients. Drawing definite conclusions about antibody kinetics or the performance of these assays in the immunocompromised is difficult due to low patient numbers, but currently very little data about antibody in the immunocompromised is available and this represents the second largest cohort of immunocompromised patients with antibody data following SARS-CoV-2 infection. Significant further work into the antibody response in the immunocompromised is still needed.

David Harrington performed the primary data analysis and literature search Tahira Azim performed the testing of all samples Mark Hopkins contributed some graphical data All authors were involved in the planning of the project and writing and editing of the manuscript