key: cord-1003367-bst7i3r0
authors: Greenland-Bews, C.; Byrne, R. L.; Owen, S. I.; Watkins, R. L.; Bengey, D.; Buist, K.; Clerkin, K.; Escadafal, C.; Finch, L. S.; Gould, S.; Giorgi, E.; Hodgkinson, A.; Mashenko, L.; Powell, D.; Savage, H. R.; Thompson, C. R.; Turtle, L.; Wardale, J.; Wooding, D.; Edwards, T.; Cubas Atienzar, A. I.; Adams, E. R.
title: Evaluation of Eight Lateral Flow Tests For The Detection Of Anti-SARS-CoV-2 Antibodies In A Vaccinated Population
date: 2022-04-05
journal: nan
DOI: 10.1101/2022.04.04.22273232
sha: 30d3df6939010403675115336a4ce68cf536dea2
doc_id: 1003367
cord_uid: bst7i3r0

With the distribution of COVID-19 vaccinations across the globe and the limited access in many countries, quick determination of an individuals antibody status could be beneficial in allocating limited vaccine doses in low- and middle-income countries (LMIC). Antibody lateral flow tests (LFTs) have potential to address this need as a quick, point of care test, they also have a use case for identifying sero-negative individuals for novel therapeutics, and for epidemiology. Here we present a proof-of-concept evaluation of eight LFT brands using sera from 95 vaccinated individuals to determine sensitivity for detecting vaccination generated antibodies. All 95 (100%) participants tested positive for anti-spike antibodies by the chemiluminescent microparticle immunoassay (CMIA) reference standard post-dose two of their SARS-CoV-2 vaccine: BNT162b2 (Pfizer/BioNTech, n=60), AZD1222 (AstraZeneca, n=31), mRNA-1273 (Moderna, n=2) and Undeclared Vaccine Brand (n=2). Sensitivity increased from dose one to dose two in six out of eight LFTs with three tests achieving 100% sensitivity at dose two in detecting anti-spike antibodies. These tests are quick, low-cost point-of-care tools that can be used without prior training to establish antibody status and may prove valuable for allocating limited vaccine doses in LMICs to ensure those in at risk groups access the protection they need. Further investigation into their performance in vaccinated peoples is required before more widespread utilisation is considered.

In the ongoing COVID-19 pandemic, the rapid development and emergency use authorisation (EUA) of multiple COVID-19 vaccines (1-3) within the first year of the SARS-CoV-2 pandemic was an unprecedented achievement. Large-scale national vaccination programmes including booster shots are wide-spread in high income countries (4, 5) . This has sparked global discussion regarding vaccine equity (6, 7) and the large disparity in the accessibility of COVID-19 vaccines between high-and lowincome countries. Determination of antibody status would be beneficial in allocating vaccine doses according to serostatus. Model-based analysis has previously indicated that this strategy has potential to optimise the impact of a COVID vaccine dose (7, 8) . Furthermore, the use of monoclonal antibody therapies (mAbs) e.g., Ronapreve for treatment of COVID-19 patients requires that patients are seronegative to be eligible for therapy and therefore require rapid determination of antibody status before treatment can begin (9).

Determination of antibody titres to a specific pathogen is commonly achieved through enzyme linked immunosorbent assay (ELISA) or similar immunoassays, which are relatively accessible in high income countries, but less accessible in low-and middle-income countries (LMICs) (10). Lateral flow tests (LFTs) however are a quick, point of care test that require minimal prior training that could be scaled up for population wide screening for presence of anti-SARS-CoV-2 antibodies. These LFTs could be used to detect anti-SARS-CoV-2 antibodies quickly and determine suitability for COVID-19 vaccination or antibody therapy. Since the COVID-19 pandemic, an enormous number of manufacturers have developed LFTs which have entered the market without standardisation, although NIBSC now have available standards for anti-SARS-CoV-2 immunoglobulin. Still, there is minimal validation procedure for these tests, and to date the available data on these tests indicates variable performance (11) (12) (13) (14) (15) (16) . Lack of consistent methodology and reference standards make comparison of results between these studies difficult. Currently the World Health Organization (WHO) only recommends the use of these tests in research settings and states that more data are required on LFT performance to determine their suitability as a tool in the COVID-19 pandemic and global vaccination programme (17).

An evaluation of multiple brands of antibody LFTs is required in vaccinated individuals at multiple time points to accurately assess their performance compared to a sensitive reference standard. To this aim we have conducted an evaluation of eight commercially available LFTs with comparisons to an automated chemiluminescent microparticle immunoassay (CMIA) that is routinely used in clinical settings. Participants were recruited from the Liverpool School of Tropical Medicine and University of Liverpool staff networks as well as members of the public through social media outreach. Participants were recruited onto an existing study (The Human Immune Responses to Acute Viral Infections study (AVIS), 16/NW/0160). All participants gave written informed consent. Healthy individuals who had received or were due to receive their COVID-19 vaccination and were aged 18 years or over were recruited to the study. Individuals taking part in COVID vaccine trials were excluded from the study. Case record form (CRF) was completed by a trained member of staff to confirm eligibility. Participants were asked to provide a blood sample at days 21 (+/-7 days), 42 (+/-7 days) post dose one and two of their COVID-19 vaccine.

Venous blood (5ml, plain serum tube) samples were collected by trained health care workers and processed on the same day of collection. Briefly, venous blood samples were centrifuged at 1500g for 10 minutes and serum was aliquoted and stored at -20C° until testing.

LFTs were performed according to the instructions for use (IFU). Serum was allowed to thaw at room temperature for 15 minutes and vortexed for 5 seconds. According to individual IFU's, 10-20µl of serum was added to the sample well and 2-3 drops of manufacturer specified buffer solution was added. Tests were run for 10-15 minutes, according to IFU, and read independently by two readers. Where there was a disagreement a third reader was used. Failed tests were repeated once. Characteristics of the of the tests used are shown in Table 1 . When no details on the antigen composition were provided in the IFU, the company was approached for further information. Although all tests used in this study detect both IgM and IgG antibodies, IgG was the focus of this investigation and results from IgM are not included.

Samples were analysed by quantitative and semi-qualitative chemiluminescent microparticle immunoassay (CMIA) on the fully automated Alinity i system (Abbot, United States) as a reference standard. SARS-CoV-2 IgG II CMIA (Abbott, Ireland) was used to quantify anti-S-RBD IgG antibodies in serum samples. To distinguish between antibodies produced from natural infection from SARS-CoV-2 to those produced from vaccination the samples were also analysed using the SARS-CoV-2 IgG I assay (Abbott, Ireland), a semi-qualitative CMIA to detect anti-nucleocapsid IgG antibodies, a method that was utilised by Narasimhan et al. and found to be effective (18) . When using these assays, individuals positive for anti-nucleocapsid IgG antibodies are considered to have been naturally infected with SARS-CoV-2 and will also test positive for anti-S-RBD IgG antibodies. If an individual gives a negative test result for anti-nucleocapsid IgG antibodies but a positive result for anti-S-RBD IgG antibodies, then this individual is assumed to have not had a natural infection and has antibodies generated in response to vaccination. Following manufacturer recommendations, results higher or equal to 50AU/ml when using the SARS-CoV-2 IgG II Quant assay were considered positive for anti-S-RBD IgG antibodies. Similarly, results higher than or equal to 1.5 S/C (Sample control index) when using the SARS-CoV-2 IgG I Quant assay were considered positive for anti-nucleocapsid IgG antibodies as per manufacturer's instructions.

A binomial mixed effect model was designed to analyse the impact of the key variables on the sensitivity of the different LFTs and to determine parameters for the calculation of the sensitivities and confidence intervals of each LFT at each dose. Details of modelling methods can be found in supplementary materials.

We used binomial mixed models to account for the clustering arising from the administration of multiple tests on the same individuals. These models were used to assess the effect of the LFT brand and other factors on the risk of a positive test. Due to the small sample size, binomial mixed models . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 5, 2022. allowed us to borrow strength information across all individuals and estimate the sensitivity of each test more reliably than the conventional approach based on simple proportions. More details on the binomial mixed models and a comparison with the standard approach for estimation of LFT sensitivity can be found in the supplementary materials. The statistical analysis was conducted in RStudio (Version: 2021.9.1.372).

A total of 95 participants were recruited and provided at least one blood sample post dose one or two. A total of 89 participants provided a sample post dose one and 69 provided a sample post dose two with 63 participants providing a sample after both dose one and two. Of the 95 participants, 63 (66.3%) were female with a mean age of 39 years. CMIA analysis was conducted on all samples and showed that seven (10.1%) individuals tested positive for anti-nucleoprotein antibodies post-dose one and six (8.7%) tested positive post-dose two. The decrease in positivity is due to an individual not providing a sample post-dose 2 rather than loss of anti-nucleoprotein antibodies between doses. Of the seven participants that tested positive, five had previously reported a positive PCR test prior to enrolment with the study. CMIA also found 88 (98.8%) samples tested positive for anti-S-RBD antibodies post dose 1 and 69 (100%) post dose 2.

Point estimates of sensitivity from the binomial mixed effect model and the standard percentage calculation were largely comparable and results from both are summarised in Six out of eight LFTs showed a statistically significant increase in sensitivity estimates from dose 1 to dose 2. This may be indicative of higher antibody titres following a second vaccine dose making it easier to detect antibodies on LFT. Fortress did not have a statistically significant increase in sensitivity however had already achieved the highest sensitivity of all eight brands at dose 1.

Sensitivities of the tests when focussing on target antigen was varied. Three tests used nucleoprotein, two used both nucleoprotein and spike and three used spike alone, with two specifying the receptor binding domain (RBD) ( Table 2 ). The tests that achieved the highest sensitivities post dose 2, CTK, Fortress and Bionote, all used different antigens of spike, spike-RBD and nucleoprotein respectively. The LFT with the lowest sensitivity was KHB which used nucleoprotein antigen.

Our mixed effect model found that vaccine brand and day of sampling (Day 21 vs day 42) had no significant effect on overall test result and were therefore removed from the model analysis (supplementary materials, Table S1 ). Dose had a significant, positive effect on positivity rate with more positive results being detected after dose 2 compared to dose 1 (Table S2) .

In this study we evaluated eight LFTs using sera from 95 vaccinated individuals, post-dose 1 and 2, to determine their sensitivity in detecting IgG antibodies specific to SARS-CoV-2 spike-RBD. We detected large variability in the sensitivities of these tests at different timepoints with Fortress having the . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 5, 2022. ;  highest sensitivity out of the eight tests evaluated, although specificity has not been considered in this study.

Overall, these results show LFTs can detect anti-S-RBD antibodies in vaccinated individuals and sensitivity increased with post-dose 2 samples. Sensitivity varies across the different brands and different antigens used. Fortress was the highest performing test in our vaccinated cohort and has also shown high sensitivity and specificity in other studies evaluating infected individuals (12, 19, 20) and is being implemented in a seroprevalence study in the UK (12, 19) . It was difficult to determine the impact the antigen in each test had on sensitivity and more information on the antigens from each brand would be beneficial for future evaluations.

Binomial mixed model analysis found that the test results were not significantly impacted by day of sample collection (Day 21 or 42 post vaccine dose) which is consistent with findings indicating IgG antibodies are detectable between 21 and 60 days after vaccination (21, 22) . Similarly, vaccine brand did not significantly impact test results, both these findings highlight that wider testing could be flexible without compromising sensitivity.

Future work should include evaluation of these tests in LMIC populations to build results for use in allocating vaccine priorities, and correlation studies to determine if a positive antibody LFT result is conducive to neutralising capacity in vaccinated individuals and if there is correlation between LFT line strength and protective antibody response. A recent review found imperfect correlation between presence of IgG and neutralising antibodies(23) however more investigation is required in this area.

LFTs have the potential to be a valuable, point of care tool to aid in the equitable distribution of vaccine doses according to existing antibody status and determine eligibility to life-saving monoclonalantibody therapies. Our study has provided an evaluation of multiple brands of LFT in vaccinated people across multiple timepoints and the variation observed in our study and other evaluations (11) (12) (13) (14) (15) highlights the importance for robust evaluation methods and standardisation to be implemented. . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 5, 2022. ; . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 5, 2022. ; https://doi.org/10.1101/2022.04.04.22273232 doi: medRxiv preprint

Let denote the test outcome for the i-th individual (1="positive", 0="negative") using the j-th brand. We then assume that the conditionally on a individual-level random effect, , follows mutually independent Bernoulli variables with probability for a positive test result (i.e.

= 1), such that

where is a brand-specific intercept and is the number of the dose with associated regression coefficient . Finally, we use 2 to denote the variance of . 

where ( | = 1) is the density function of the individual-level random effects conditioned to having observed a positive reference test and [ = 1 | ]is the probability of a positive test based on the k-th brand, as defined in [1] . To compute the integral in [2] we use a quadrature approach to carry out numerical integration.

To compute the confidence intervals of the sensitivity estimate in [2] , we proceed as follows.

1. We simulate 1000 samples for and 2 from a Multivariate Gassian distribution with mean given by the maximum likelihood estimates of and 2 , and covariance matrix given by the inverse of the negative of the observed Fisher information. 2. For each of the 1000 samples simulated in the previous step, we compute the corresponding estimates of the sensitivity according to [2] . 3. Using the simulated sensitivity values from the previous step, we compute the 0.025 and 0.975 quantiles to obtain a 95% confidence interval.

. CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted April 5, 2022. ; . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted April 5, 2022. ; https://doi.org/10.1101/2022.04.04.22273232 doi: medRxiv preprint

Point estimates of LFT sensitivity from the binomial mixed model analysis were validated against proportional sensitivity that was calculated in Excel 2019 (Microsoft 365). Results from this validation are shown in Table S2 and S3 

Regulatory approval of Spikevax (formerly COVID-19 Vaccine Moderna) -GOV.UK [Internet

Regulatory approval of Pfizer/BioNTech vaccine for COVID-19 -GOV.UK [Internet

Regulatory approval of Vaxzevria (previously COVID-19 Vaccine AstraZeneca) -GOV.UK [Internet

COVID-19 vaccination: a guide to booster vaccination for individuals aged 40 years and over -GOV.UK [Internet

COVID-19 Vaccine Booster Shots | CDC

Why There Should Be a Moratorium on COVID-19 Booster Shots Until Low-Income Countries Get Vaccinated

Covid-19: People who have had infection might only need one dose of mRNA vaccine

National Insitute for Health and Care Excellence (NICE). NICE updates managing COVID guideline with new monoclonal antibody recommendations | News and features | News | NICE

Evaluation of Serological SARS-CoV-2 Lateral Flow Assays for Rapid Point-of-Care Testing

Clinical and laboratory evaluation of SARS-CoV-2 lateral flow assays for use in a national COVID-19 seroprevalence survey

Rapid antibody testing for SARS-CoV-2 vaccine response in pediatric healthcare workers

COVID19 antibody detection using lateral flow assay tests in a cohort of convalescent plasma donors

Diagnostic performance of seven rapid IgG/IgM antibody tests and the Euroimmun IgA/IgG ELISA in COVID-19 patients

Clinical evaluation of the abbott alinity sars-cov-2 spike-specific quantitative igg and igm assays among infected, recovered, and vaccinated groups

SARS-CoV-2 lateral flow assays for possible use in national covid-19 seroprevalence surveys (React 2): diagnostic accuracy study

Comparative evaluation of ten lateral flow immunoassays to detect SARS-CoV-2 antibodies

SARS-CoV-2 Antibody Rapid Tests: Valuable Epidemiological Tools in Challenging Settings. Microbiology Spectrum

Antibody responses to SARS-CoV-2 vaccines in 45,965 adults from the general population of the United Kingdom

Prevalence of neutralising antibodies against SARS-CoV-2 in acute infection and convalescence: A systematic review and meta-analysis

Onsite COVID-19 IgG/IgM Rapid Test

NowCheck COVID-19 IgM/IgG Test

Edinburgh Genetics COVID-19

Colloidal Gold Immunoassay Testing Kit, IgG/IgM Combined

Diagnostic Kit for SARS-CoV-2

SARS-CoV-2 IgM/IgG Ab Rapid Test