key: cord-0856978-okq47xqc
authors: Irwin, Natalie; Murray, Lyle; Ozynski, Benjamin; Richards, Guy A; Paget, Graham; Venturas, Jacqueline; Kalla, Ismail; Diana, Nina; Mahomed, Adam; Zamparini, Jarrod
title: nnAge significantly influences the sensitivity of SARS-CoV-2 rapid antibody assays
date: 2021-07-13
journal: Int J Infect Dis
DOI: 10.1016/j.ijid.2021.07.027
sha: 7b377f37c8cbaed710a03d96934cdf11884b0268
doc_id: 856978
cord_uid: okq47xqc

BACKGROUND: Point of care serological assays are a promising tool in COVID-19 diagnostics but do have limitations. This study evaluated the sensitivity of five rapid antibody assays and explored factors influencing their sensitivity to detect SARS-CoV-2-specific IgG and IgM antibodies. METHODS: Finger-prick blood samples from 102 participants, within two to six weeks of PCR-confirmed COVID-19 diagnosis, were tested for IgG and IgM on five rapid serological assays. The assay sensitivities were compared, and patient factors evaluated in order to investigate potential associations with assay sensitivity. RESULTS: Sensitivity ranged from 36% to 69% for IgG and 13% to 67% for IgM. Age was the only factor significantly influencing the likelihood of a detectable IgG or IgM response. Individuals aged 40 years and older had an increased likelihood of a detectable IgG or IgM antibody response by rapid antibody assay. CONCLUSION: Rapid serological assays demonstrate significant variability when used in a real-world clinical context. There may be limitations in their use for COVID-19 diagnosis amongst the young.

and immunoglobulin M (IgM) antibodies to the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) on a venous or finger-prick whole blood sample without the need for specialized equipment. Such assays are useful for rapid antibody testing in surveillance programmes in outbreak settings or in high seroprevalence areas. The assays may assist in the diagnosis of suspected COVID-19 in patients who test negative for SARS-CoV-2 by polymerase chain reaction (PCR) on naso-or oropharyngeal swabs. (Theel, 2020) In addition, they require minimal operator training and have a turnaround time of under thirty minutes. (Riccò, 2020) Some valid concerns about the performance quality of these rapid assays exist, and most available rapid assays have been subjected only to single centre internal validation studies, in small populations. (Department of Health, 2020) Furthermore, the threshold antibody titre required to generate a detectable result on these devices is poorly described. Reported overall IgG/IgM sensitivities range from 18.4 to 93.3% and vary depending on disease severity and duration since symptom onset. (Riccò, 2020) This study critically evaluated the sensitivity of five rapid antibody assays for detection of SARS-CoV-2-specific IgM and IgG antibodies, on finger-prick blood samples amongst patients with COVID-19 confirmed by PCR on nasopharyngeal or oropharyngeal swab. Importantly, this study also investigates patient factors that influence the sensitivity of such assays. A single drop (10-20μL) of whole blood drawn by fingertip puncture was deposited in the sample well of each test cassette. Two to five drops of reagent buffer were then added to the sample well and results read 15-20 minutes later according to the specific manufacturer's instructions. Assays three and five had not previously been validated on finger-prick specimens. Two readers (N.I. and/or J.Z. and/or B.O.) read the cassettes with the naked eye and a third reader (J.V.) settled any disputes. A test was considered valid if a control line was visualized and was interpreted as positive if 6 the control line and the line for IgG or IgM or both was seen. Each of the five kits were evaluated once on each participant with no replication of testing on any participants. All test cassettes and reagents for each of the five different kits were from the same manufacturing batch.

Data were analysed using Prism 8.4.3 (GraphPad Software Inc, La Jolla, California) using standard non-parametric statistical tests as appropriate. Continuous data were expressed as medians with interquartile ranges (IQRs) and categorical variables presented as numbers and percentages. Fisher's exact tests were used to compare results in the age, time and severity groups and Spearman's correlation co-efficients were used to assess agreement between test assays. The multivariate logistic regression analysis was performed using IBM SPSS Statistics 26.0 (IBM Corporation, Armonk, New York).

We conducted antibody testing using all five rapid antibody assays on 102 participants with previous PCR-confirmed . The majority of our participants were female (61%) and of black ethnicity (40%). Median age in the cohort was 37.5 years (IQR 29 -45.75) and most of the participants were HCWs (68%). The majority of our cohort were tested between 15-and 42-days post-positive PCR testing (n=83, 81%) with 4 (4%) and 15 (15%) being tested less than 15 days and more than 42 days post-PCR-testing, respectively. Comorbidities amongst the participants included hypertension (n=14, 14%), diabetes (n=12, 12%), asthma (n=10, 10%), HIV (n=6, 6%) and cancer (n=3, 3%). A small number of the patients in our cohort required inpatient management for COVID-19 (n=18, 18%), in keeping with the number of participants classified as having severe disease. Additional demographic data are presented in Table 1 .

Overall sensitivity to detect SARS-CoV-2-specific IgG and IgM antibodies was below 70% in all assays ( Figure 1 ). IgG sensitivity ranged from 36% (Innovita) to 69% (Dynamiker) whilst IgM sensitivity ranged from 13% (Innovita) to 67% (Dynamiker).

Of note, the sensitivities of the Innovita (13%), AllTest (15%) and Altis (26%) assays to detect IgM were markedly lower than those of the Dynamiker (67%) and Cellex (64%) assays.

Variability in diagnostic accuracy between the five rapid antibody assays for the detection IgG and/or IgM was assessed through the use of a correlation matrix represented in a heatmap ( Figure 2 ). This measured agreement between each assay for the detection of IgG or IgM. Four of the five assays (Dynamiker, AllTest, Altis and Cellex, Figure 2 ; r >0.8 for all) correlated well with each other in the detection of IgG. However, none of these four assays correlated well with the Innovita assay for IgG. Strong correlation for IgM results was found between only two of the five assays (Dynamiker and Cellex, Figure 2 ; r >0.8). The correlation results were all statistically significant at p<0.05.

Analysis of the factors associated with improved sensitivity of the rapid antibody assays The participant factors potentially associated with the sensitivity for detection of IgG or IgM by rapid antibody assay were investigated and included: gender, time since positive SARS-CoV-2 PCR test (≤30 days vs >30 days), age ( <40 years vs ≥40 years) and COVID-19 disease severity (asymptomatic-mild vs moderate-severe).

Most strikingly, there was a significant reduction in IgG sensitivity for participants under 40 years of age compared to those over 40 years for all of the assays used ( for all except the AllTest). This is in keeping with a study by Wu et al which showed significantly higher neutralising antibody titres in elderly and middle-aged patients when compared to young patients. (Wu F, 2020) Gender Male gender has previously been associated with COVID-19 severity. (Peckham, 2020 ) However, we did not find any difference in the sensitivity of any of the rapid antibody assays for IgG or IgM based on gender within our cohort (Tables 2 and 3) .

We hypothesized that the dynamic antibody responses during and after COVID-19 may influence the detection of IgG and IgM SARS-CoV-2-specific antibodies. The association between time since the SARS-CoV-2 PCR test and the detection of IgG and IgM antibodies was explored. There was no difference in IgG sensitivity between those presenting within 30 days of a positive PCR test compared to those presenting after 30 days as shown in Table 2 . However, a significantly lower proportion of individuals tested more than 30 days after a positive SARS-CoV-2 PCR test had a detectable IgM response by the AllTest assay (23% (≤30 days) vs 6% (>30 days), p = 0.02, OR 0.2228(0.0644 -0.7993), Table 3 ). There was no significant difference in IgM sensitivity in the two time groups for the other four rapid antibody assays (p>0.05 for Dynamiker, Innovita, Altis and Cellex, Table 3 ).

In order to better define the temporal relationship between time elapsed since COVID-19 and the sensitivity of the rapid assays to detect SARS-CoV-2-specific IgG and IgM antibodies, we further stratified individuals by time since positive SARS-CoV-2 PCR test and compared the sensitivity of the assays between these strata ( Figure 3 ). There was great heterogeneity in the sensitivity to detect IgM between assays and no consistent pattern of sensitivity of assays according to time since positive PCR test. Contrasting patterns of sensitivity for IgM by time are illustrated by the Dynamiker and AllTest assays. There was a significantly higher sensitivity for detection of IgM in individuals testing more than 40 days after a positive PCR test when compared to those testing less than 21 days when using the Dynamiker assay (88.9% (>40 days) vs 64% (<21 days), p=0.046), Figure 3 ). By contrast, when using the AllTest assay the sensitivity for IgM was lower in the group testing more than 40 days after a positive PCR test (0% (>40 days) vs 24% (<21 days), p=0.032, Figure   3 ). For IgG there was a trend towards increasing sensitivity associated with increased time since positive PCR test for all of the rapid assays. This was significant for the Dynamiker and Cellex assays where sensitivities were higher in individuals testing more than 40 days after a positive PCR test when compared to those testing 21-30 days thereafter (Dynamiker: 88.9% (>40 days) vs 60.7% (21-30 days), p=0.049; Cellex: 88.9% (>40 days) vs 60.7% (21-30 days), p=0.049, Figure   3 ).

COVID-19 disease severity has been shown to impact the magnitude of the SARS-CoV-2-specific antibody response. We therefore investigated whether disease severity had an influence on the sensitivities of the rapid antibody assays to detect IgG or IgM antibodies to SARS-CoV-2. Interestingly, there was no significant difference in IgG sensitivity for those with asymptomatic or mild disease compared to those with moderate or severe disease between the two groups for each assay individually (Table 2 ). There was also no significant difference in IgM sensitivity between the two groups (p>0.05 for all, Table 3 )

A multivariate logistic regression was performed to exclude confounding variables and to confirm the factors associated with an increased likelihood of a detectable SARS-CoV-2-specific IgG or IgM response by rapid antibody assay. The variables included age, gender, time since positive test and severity. Age >40 years was the only variable associated with a significantly increased likelihood of a detectable IgG and IgM response by rapid antibody testing ( Supplementary Tables 1 and 2) . This was significant for all of the rapid antibody assays for IgG and all except the AllTest assay for IgM (Supplementary Tables 1 and 2) .

Rapid serological assays are increasingly becoming an essential component of surveillance of outbreaks and retrospective diagnoses of COVID-19. These assays have appeal as they are rapid, inexpensive and user-friendly. However, data evaluating their true clinical performance on finger-prick analyses at the POC, in a real-world clinical context, is urgently needed.

In this present study, tested against the reference gold standard PCR, the sensitivity of five rapid antibody assays ranged from 36% to 69% for IgG and 13% to 67% for IgM. This is significantly lower than that reported in previous validation studies of these rapid tests.(Dynamiker Biotechnology (Tianjin) Co., Ltd., 2020; Mølbak, 2020;

Lassaunière, 2020) The South African Health Products Regulatory Authority (SAHPRA) specifies that a minimum clinical sensitivity of 85% within 95% confidence intervals be proven prior to registration.(SAHPRA, 2020) All of these rapid antibody assays fall below that benchmark when used in our cohort. The wide spread of the results of our sensitivity analysis are comparable with findings of existing studies. In a pooled analysis of 1030 POC antibody assays, by Riccò et al, combined sensitivity for IgG and IgM, ranged from 18.4% to 93.3% and with an average of 64.8% (95%CI 54.5-74.0). (Riccò, 2020) We also report poor correlation between the results of the five assays in comparison to each other, particularly when testing for IgM. This finding has also been previously described, and prompted Van Elslande et al to question the need for measuring IgM SARS-CoV-2 antibodies at all. (Cassaniti, 2020;  Van Elslande, 2020) The heterogenous clinical sensitivity performance of IgM in the antibody assays in our study further suggest limited clinical utility. Given the poor correlation between the Innovita and all other kits, we would not recommend use of this kit on fingerprick blood samples.

Our analysis of patient factors associated with an improved sensitivity of the rapid antibody tests revealed an association between age and sensitivity. All of the assays demonstrated improved sensitivities in those patients aged >40 years for IgG and all the assays except the Altis for IgM. The higher sensitivity of these tests in older participants may indicate higher antibody titres in these individuals and therefore a higher likelihood of detection of such antibody responses by the rapid assays. Older patients with COVID-19 have been shown to have higher SARS-CoV-2-specific IgG and IgM antibody titres than younger patients, although the reason for this is unclear. (Huang, 2020; Wu F, 2020) It is likely that increased SARS-CoV-2-specific antibody titres correlate with more severe COVID-19, and that increased age has been correlated with severe disease and worse outcome. Zhou, 2020) .

When both age and disease severity were included in a multivariate logistic regression analysis, only age was associated with an increased likelihood of a positive rapid IgG or IgM antibody response, thereby suggesting that age may play a role independently of its association with disease severity. Our cohort primarily consisted of participants that had mild COVID-19 with 80% of the cohort in the mild to moderate category. Due to the fact that younger individuals typically experience milder COVID-19 disease and have an increased rate of asymptomatic infection, our findings may suggest a significant impediment to the use of these assays to assess seroprevalence in younger participants.

The impact of time since diagnosis of COVID-19 by positive PCR test and the sensitivity of the rapid assays is intriguing. The trend towards increased sensitivity to detect IgG and increased time since positive PCR is as expected and is consistent with previous studies. (Deeks, 2020; Wu JL, 2020) For IgM, the two kits with the highest sensitivity to detect IgM (Dynamiker and Cellex) showed highest sensitivity when tested amongst individuals more than 40 days post PCR test. Although generally believed to be relatively short-lived, SARS-CoV-2-specific IgM antibodies have been shown to persist for up to two and a half months after SARS-CoV-2 infection and therefore may explain this. (Iyer, 2020) It is not clear whether poor IgM sensitivity in some of the assays is possibly associated with higher specificity and further assessment, possibly with the use of ELISA assays, would be required to investigate this.

There are limitations to this study. Although used as the reference gold standard for the diagnosis of COVID-19, PCR-based testing gives no indication of patient seropositivity. The negative serological results reported here may represent low antibody titres or low participant seroconversion, rather than a failure of the test to detect antibodies. Furthermore, two assays (Innovita and Cellex) that were included have not previously been validated on finger-prick blood samples. Although we consider the effect is likely to be small, as all other tests could be used across blood sample platforms, this may have contributed to their poor performance. Owing to inadequate access to negative control samples, we were not able to perform our own corresponding specificity analysis, however, a Cochrane Diagnostic Test Accuracy Review has shown such rapid serological assays to have very high specificity. (Deeks, 2020) 

This study described an overall underperformance of rapid serological assays to detect an IgG and IgM response two to six weeks after PCR confirmed SARS-CoV-2 infection. We highlight the heterogenous ability of the antibody assays to detect IgM and describe a significant independent association between age >40 years and increased sensitivity for IgG and IgM seropositivity. Judicious clinical use, recognising the limitations of rapid serological assays, especially amongst the young, is necessary. Lancet. 2020 Mar 28; 395(10229):1054 -1062 . doi: 10.1016 30566-3. 

Rapid Test is inadequate for diagnosis of COVID-19 in acute patients referring to emergency room department

Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev

Guidance on the use Of SARS CoV-2 Antibody Tests. SA Coronavirus Online Portal

2019-nCOV IgG/IgM Rapid Test Package Insert. Available at: www.dynamiker

Temporal antibody responses to SARS-CoV-2 in patients of coronavirus disease 2019

Persistence and decay of human antibody responses to the receptor binding domain of SARS-CoV-2 spike protein in COVID-19 patients

Evaluation of nine commercial SARS-CoV-2 immunoassays. Infectious Diseases (except HIV/AIDS)

Performance Evaluation of the 2019-nCOV IgG/IgM Rapid Test From Dynamiker Biotechnology (Tianjin) Co., Ltd

 National Institutes of Health. COVID-19 Treatment Guidelines Panel

COVID-19) Treatment Guidelines

Male sex identified by global COVID-19 meta-analysis as a risk factor for death and ITU admission

Point-of-Care Diagnostic Tests for Detecting SARS-CoV-2 Antibodies: A Systematic Review and Meta-Analysis of Real-World Data

MD007: Specification criteria for COVID-19 serological test kits

The Role of Antibody Testing for SARS-CoV-2: Is There One?

Diagnostic performance of seven rapid IgG/IgM antibody tests and the Euroimmun IgA/IgG ELISA in COVID-19 patients

Evaluating the Association of Clinical Characteristics with Neutralizing Antibody Levels in Patients Who Have Recovered from Mild COVID-19 in

Four point-of-care lateral flow immunoassays for diagnosis of COVID-19 and for assessing dynamics of antibody responses to SARS-CoV-2

Antibody Responses to SARS-CoV-2 in Patients With Novel Coronavirus Disease

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan

The authors declare no conflict of interest

First Medical Company, Cape Town; SMD Technologies, Johannesburg and Altis Biologics, Pretoria donated test assays for use in this study but had no role in the study design, data collection, data analysis and/or writing of the manuscript

This data has not been previously presented at any meetings or conferences