key: cord-0763331-wcqa0f46
authors: Davies, B.; Araghi, M.; Moshe, M.; Gao, H.; Bennet, K.; Jenkins, J.; Atchison, C.; Darzi, A.; Ashby, D.; Riley, S.; Barclay, W.; Elliott, P.; Ward, H.; Cooke, G.
title: Acceptability, usability and performance of lateral flow immunoassay tests for SARS-CoV-2 antibodies: REACT-2 study of self-testing in non-healthcare key workers
date: 2021-06-25
journal: nan
DOI: 10.1101/2021.06.21.21259254
sha: 22339351f9528f808e8d8a93d47886e16ddf2b3a
doc_id: 763331
cord_uid: wcqa0f46

Background Seroprevalence studies in key worker populations are essential to understand the epidemiology of SARS-CoV-2. Various technologies, including laboratory assays and point-of-care self-tests, are available for antibody testing. The interpretation of seroprevalence studies requires comparative data on the performance of antibody tests. Methods In June 2020, current and former members of the UK Police forces and Fire service performed a self-test lateral flow immunoassay (LFIA) and provided a saliva sample, nasopharyngeal swab, venous blood samples for Abbott ELISA and had a nurse performed LFIA. We present the prevalence of PCR positivity and antibodies to SARS-CoV-2 in this cohort following the first wave of infection in England; the acceptability and usability of self-test LFIAs (defined as use of the LFIA kit and provision of a valid result, respectively); and determine the sensitivity and specificity of LFIAs compared to laboratory ELISAs. Results In this cohort of non-healthcare key workers, 7.4% (396/5,348; 95% CI, 6.7-8.1) were antibody positive. Seroprevalence was 8.9% (6.9-11.4) in those under 40 years, 11.5% (8.8-15.0) in those of non-white British ethnicity and 7.8% (7.1-8.7) in those currently working. The self-test LFIA had an acceptability of 97.7% and a usability of 90.0%. There was substantial agreement between within-participant LFIA results (kappa 0.80; 0.77-0.83). The LFIAs (self-test and nurse-performed) had a similar performance: compared to ELISA, sensitivity was 82.1% (77.7-86.0) self-test and 76.4% (71.9-80.5) nurse-performed with specificity of 97.8% (97.3-98.2) and 98.5% (98.1-98.8) respectively. Conclusion A greater proportion of the non-healthcare key worker cohort showed evidence of previous infection with SARS-CoV-2 than the general population at 6.0% (5.8-6.1) following the first wave in England. The high acceptability and usability reported by participants and the similar performance of self-test and nurse-performed LFIAs indicate that the self-test LFIA is fit for purpose for home-testing in occupational and community prevalence studies.

During the response to the COVID-19 pandemic in the United Kingdom (UK), people who, due to the nature of their work, were unable to work from home were known as "key workers" [1] . The UK key worker population is estimated to be 10.6 million people (33% of the total workforce) with the majority (69%) working in industries other than health and social care, including education and childcare, key public services, transport industry and food sector [1, 2] . It is well established that key workers are at increased risk from infection by SARS-CoV-2 and subsequent COVID-19 [3, 4] .

In April 2020 the UK Government initiated a programme of COVID-19 testing which included studies of the prevalence of antibodies to SARS-CoV-2 in the community [5] . In order to facilitate widespread community testing, the REal Time Assessment of Community Transmission-2 (REACT-2) study undertook a programme of development work using selftest finger prick lateral flow immunoassays (LFIAs) for the detection of antibodies to SARS-CoV-2. The initial laboratory validation and usability of the test was carried out among healthcare professionals [6] , and further usability and acceptability studies were conducted among a random sample of adults in the population [7] .

Here, as part of the REACT-2 programme [8] , we aimed both to quantify the prevalence of SARS-CoV-2 (current and past infection) among a subset of public-facing non-healthcare key workers following the first wave of infection in England, and to test at scale the performance of self-test LFIAs versus nurse-administered LFIAs. This was done to determine the acceptability, usability and performance of self-test point-of-care LFIAs in key worker occupations who do not have specialist training in the use of medical devices. Finally, we evaluated the performance of the LFIA against an established ELISA antibody test in sera.

Non-healthcare key worker participants were identified through an established occupational research cohort (Airwave Health Monitoring Study), a longitudinal occupational cohort of over 50,000 people who work, or have worked, for the Police forces in the UK [9] and a register of current employees of the West Midlands (UK) Fire Service. All participants with a . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted June 25, 2021. ; https://doi.org/10.1101/2021.06.21.21259254 doi: medRxiv preprint residential address within a reasonable distance (approximately 50 kilometres) of one of six testing sites (Bournemouth, Derby, Keele, London, Manchester and Warwick) and no known medical condition that might increase bleeding risk were sent a letter or email inviting them to take part together with a participant information sheet. Those who did not respond were sent a reminder at 4 weeks and recruitment was capped at 5,500 participants [8] .

Participants were recruited between 1 June and 10 July 2020. Social distancing (except during finger-prick and venepuncture) and personal protective equipment were implemented at the clinics. Consent was obtained from participants who had been symptom-free for the previous 7-days and they were provided with instructions, a set of study ID stickers (barcode), a self-test LFIA kit, a nasopharyngeal kit and a saliva kit.

Participants were then invited to conduct the LFIA ("self-test") by following the written/visual instructions provided [10] . No direct instruction was provided from staff at the clinic facility.

The LFIA (Fortress, Northern Ireland) used tests antibody against the spike ("S") protein. It was previously evaluated as having sensitivity 84% (95% CI, 70.5%, 93.5%) and specificity 98.6% (97.1%, 99.4%) [6] . While participants were waiting the 10-15 minutes to read the LFIA test they performed a self-administered saliva collection (2ml, using Oragene kit [DNA Genotek, Canada] and drool technique) using the instructions provided. After reading the LFIA result, participants uploaded an image of the completed self-test LFIA. Participants then underwent a repeat finger-prick LFIA administered by a health care professional ("nurse-performed") and provided a set of venous blood samples. After leaving the building, participants completed a self-administered nasopharyngeal swab in an outside shelter area or their private vehicle, following the instructions and video, and then placed the sample in the "drop box" at reception. Participants were also asked to complete a brief questionnaire, which included questions on demographic characteristics, occupation, health status (including COVID-19), usability and acceptability of the LFIA device [7] .

Samples were transported daily via courier to laboratories for processing. The saliva sample and nasopharyngeal swab were analysed by RT-PCR for SARS-CoV-2 and participants were contacted (email or phone) with the results within three days and, if positive, provided with the latest government instructions on self-and household isolation. Participants had a venous blood sample analysed by ELISA (Abbott, USA) for SARS-CoV-2 IgG antibody. We applied the manufacture recommended diagnostic cut-off (binding ratio of 1.4) to determine . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted June 25, 2021. ; https://doi.org/10.1101/2021.06.21.21259254 doi: medRxiv preprint a positive result from this quantitative immunoassay [11] . Participants received the results of the blood tests within 4 weeks via letter.

We define the acceptability of the self-test LFIA as the proportion of participants attending the test centre who consented to, and used, the provided kit [7] . We define the usability of the self-test LFIA as the proportion of participants that used the kit who achieved a valid result. A positive LFIA test was defined as IgG positive, and a negative test was defined as IgG negative, regardless of IgM result [7] . For those that did not complete the test, we present the reason given.

We excluded people without valid questionnaire responses from relevant analyses and retained tests with a positive or negative result. We estimated the prevalence of RT-PCR positivity from nasopharyngeal swab and saliva samples, and antibody positivity was defined as a positive Abbott ELISA test on serum.

After excluding LFIAs with an invalid result, we report the concordance of self-test to nurseperformed LFIA results using the Cohen's kappa statistic: <0, poor agreement; 0.00-0.20, slight agreement; 0.21-0.40, fair agreement; 0.41-0.60, moderate agreement; 0.61-0.80, substantial agreement; and >0.8, almost perfect agreement [12] . For participants with a valid LFIA (participant and nurse) and Abbott ELISA result, we determined the positive (sensitivity) and negative (specificity) percent agreement of the LFIA tests compared to the serum Abbott ELISA test.

Finally, we undertook an independent visual inspection of the uploaded images from the discordant LFIA test pairs (self-test and nurse-performed) and a similar-sized sample of concordant pairs, to investigate reasons for any differences. Two reviewers (BD and MA) independently reviewed the discordant images and a third (HW) was consulted to adjudicate on and resolve any differences. One reviewer (BD) reviewed the concordant images.

We used logistic regression to quantify associations between demographic characteristics and i) detected antibody status (positive Abbott ELISA test) and ii) not having a valid self-test LFIA result. 95% confidence intervals were calculated using the Wilson's method. Analyses were conducted in STATA version 13.1 or higher (StataCorp, College Station, TX, USA).

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted June 25, 2021. Airwave study participants have given consent to be contacted for other research studies (IRAS project ID: 259978).

Study recruitment was capped at 5,500, therefore of the 24,182 individuals invited to the participate in the study, 5,554 (23.0%) booked into clinic and of these 5,453 (98.2%) attended (Fig. 1) . The majority (88%, n=4,729) of participants were 40 years and older, 65% (n=3,483) were men, 92% (n=4,882) were white British (Table 1) 

IgG antibodies to SARS-CoV-2 were found in 396 of 5,348 participants using the Abbott ELISA, a prevalence of 7.4% (6.7, 8.1). Prevalence varied by age being highest in people under 40 years of age (8.9%; 6.9,11.4) and lowest in people over 60 years (5.3%; 3.9, 7.0); by ethnicity, being higher in non-white (11.5%; 8.8, 15.0) compared to white (7.0%; 6.3, 7.7) participants; employment status, being higher in people currently working (7.8%; 7.1, 8.7) compared to people not currently working (4.8%; 3.5, 6.5) and region: higher in London (11.7%; 9.9, 13.8) compared to other regions except North West and lowest in the South West (3.0%; 2.0, 4.5) ( Table S2 ). The review showed that participants were more likely to report negative tests as positive, and nurses were more likely to miss faint positives, which may explain the lower sensitivity of nurse-tests compared to Abbott ELISA (Supplementary Table   S3 ). Review of the uploaded images for the sample of 145 concordant self-test and nurseperformed tests found that 129/129 available image pairs were concordant (Supplementary Table S4 ).

The self-test LFIA test was attempted by 5,328 participants, an acceptability of 97.7%. Of the participants who attempted the test, 4,797 obtained a valid result, giving a usability of 90.0%.

The most commonly cited reasons for not successfully completing the LFIA were "I could not read the result" (n=78), "It was too fiddly for me to manage" (n=51) and "I did not manage to get the buffer onto the test" (n=37). An invalid self-test LFIA result was 20% less likely for women compared to men (OR 0.78; 0.63, 0.97) and 40% more likely for people who were not currently working compared to people currently working (OR 1.38; 1.03, 1.85) (Supplementary Table S5 ). The majority of participants (88.5%, n=4,638) were "very confident" that they had interpreted their results correctly. Of the fifty participants who provided a preference for where to perform the test, 82.0% (62.9, 90.2) stated home.

One in 13 people (7.4%; 6.7, 8.1) in this non-healthcare key worker cohort had evidence of previous infection with SARS-CoV-2 following the first wave of the epidemic in England, . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted June 25, 2021. ; https://doi.org/10.1101/2021.06.21.21259254 doi: medRxiv preprint rising to one in nine in London (11.7%; 9.9, 13.8). This prevalence was higher than in a representative sample of the population at the time at 6.0% (5.8, 6.1), but lower than in key workers in healthcare at 11.7% (10.5, 13.1) [4] . The participants reported a high acceptability and usability of self-test LFIAs, with 90% of participants obtaining a valid antibody test result.

There was no significant difference in the performance of self-test and nurse-performed LFIAs. The prevalence of infection in the cohort was low at 0.07% (n=4) on RT-PCR from nasopharyngeal swab, and none of the saliva samples tested positive. This low prevalence reflects the declining epidemic at the time of the study, during June to July 2020 [13, 14] .

The Fortress LFIA used in this study followed a rigorous evaluation of five commercially available LFIAs [6] . We used the Abbott ELISA as the primary reference standard. An evaluation by Public Health England (PHE), found that it met the manufacturer-reported performance for specificity, although not for sensitivity [11] .

Our study provides further evidence for the association of occupation, involving key workers in a public-facing role, with risk of infection by SARS-CoV-2. Previous research suggests that in mid-June to mid-July, the crude prevalence in key workers (excluding health and care-home workers) in England was 6.1% (5.7, 6.4) [15] , over a percentage point lower than the 7.4% (6.7, 8.1) reported here. However, it is possible that the prevalence following the first wave of the pandemic among participants in our study could be under-estimated given uncertainty about antibody waning and the duration of a detectable antibody response [16] .

We found similar demographic patterns in prevalence to those seen in the national REACT-2 study, where the highest prevalence was also found in the London region, in people of nonwhite ethnicity and younger adults [4] .

The sensitivity of self-test LFIAs estimated against Abbott ELISA in this cohort of nonhealthcare key workers was 82.1% (77.8, 85.8), similar to previously reported sensitivity of 84% (70.5, 93.5) in a cohort of healthcare workers with PCR-confirmed COVID-19 [6] . The specificity against Abbott ELISA in this study was 97.8% (97.3, 98.2), similar to the 98.6% (97.1, 99.4) using pre-pandemic sera in the laboratory [6] .

Overall, there was substantial agreement between self-test to nurse-performed LFIAs, with only 2.8% having discrepant results. There was no evidence that nurse-performed tests were better than self-tests, although there was a suggestion that participants were more likely to record a test as positive and healthcare practitioners were more likely to report as negative. This was supported by the visual reinspection of the small number of discordant pairs of LFIAs. The study suggests that there is no gain in accuracy to be had by committing . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted June 25, 2021. ; https://doi.org/10.1101/2021.06.21.21259254 doi: medRxiv preprint extra resource to obtain healthcare practitioner performed LFIAs in large-scale community surveys and that self-test is a viable and acceptable approach.

We aimed to replicate the experience of completing a home self-test LFIA as a means of obtaining prevalence of previous SARS-CoV-2 infection in non-healthcare occupational settings by providing participants with no additional assistance, beyond the test instruction materials. The idea was that, if successful, this could then be extended to obtain prevalence estimates in the wider population at low cost, without the need for supervision by a healthcare practitioner. We found little difference between the results of LFIAs administered as a self-test or by a nurse, supporting the unsupervised use of the LFIA. However, our results might not be fully generalisable. The majority of participants were recruited from the Airwave occupational cohort [9] . The cohort is not demographically (age, sex, ethnicity, geography) representative of the adult population of England, participants had to be able to travel to a test site (mostly in a private vehicle) and, being an occupational cohort, participants were healthier than the general population ('healthy cohort' effect [9] ).

Participants (emergency service staff) may also have been more familiar with medical procedures, which may have had an impact on the generalisability of the usability and acceptability findings to the general population. Nonetheless, parallel studies carried out in the community at around the same time provided similar results on usability and acceptability (although estimates of prevalence were lower, reflecting greater exposure of public-facing workers to risk of infection) [7] . We compared results of the self-test and nurse-performed LFIA with a quantitative ELISA test (Abbott) using an established cut-point to denote positivity. To the extent that the "gold standard" ELISA test itself does not have perfect sensitivity and specificity [11] , will have introduced some error into those comparisons. Participants reported a high acceptability and usability for the LFIAs and there was little . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted June 25, 2021. 

for National Statistics. Official Statistics: Key workers: population and characteristics

Guidance: Critical workers who can access schools or educational settings

Office for National Statistics. Coronavirus and key workers in the UK. Available at

SARS-CoV-2 antibody prevalence in England following the first peak of the pandemic

Policy paper: Coronavirus (COVID-19): Scaling up our testing programmes

Clinical and laboratory evaluation of SARS-CoV-2 lateral flow assays for use in a national COVID-19 seroprevalence survey

Usability and acceptability of home-based self-testing for SARS-CoV-2 antibodies for population surveillance

REal-time Assessment of Community Transmission (REACT) of SARS-CoV-2 virus: Study protocol

The Airwave Health Monitoring Study of police officers and staff in Great Britain: Rationale, design and methods

Your at-home antibody test for coronavirus

Evaluation of the Abbott SARS-CoV-2 IgG for the detection of anti-SARS-CoV-2 antibodies

The Measurement of Observer Agreement for Categorical Data

Community prevalence of SARS-CoV-2 virus in England during

Transient dynamics of SARS-CoV-2 as England exited national lockdown

The duration, dynamics and determinants of SARS-CoV-2 antibody responses in individual healthcare workers

Prevalence of antibody positivity to SARS-CoV-2 following the first peak of infection in England: Serial cross-sectional studies of 365,000 adults

We thank key collaborators on this research: Department of Epidemiology and Biostatistics at Imperial College: Eric Johnson and Rob Elliott; Institute of Global Health Innovation at Imperial College: Gianluca Fontana, Sutha Satkunarajah and Lenny Naar; Airwave study: Andy Heard, Paul Downey, Antoinette Amuzu.

We declare no competing interests.