key: cord-0990467-r3hqpixr
authors: Mistry, Dylan A.; Wang, Jenny Y.; Moeser, Mika-Erik; Starkey, Thomas; Lee, Lennard Y. W.
title: A systematic review of the sensitivity and specificity of lateral flow devices in the detection of SARS-CoV-2
date: 2021-08-18
journal: BMC Infect Dis
DOI: 10.1186/s12879-021-06528-3
sha: 1c2e78700ce40dfbea6d4261c40210b9e2c74247
doc_id: 990467
cord_uid: r3hqpixr

BACKGROUND: Lateral flow devices (LFDs) are viral antigen tests for the detection of SARS-CoV-2 that produce a rapid result, are inexpensive and easy to operate. They have been advocated for use by the World Health Organisation to help control outbreaks and break the chain of transmission of COVID-19 infections. There are now several studies assessing their accuracy but as yet no systematic review. Our aims were to assess the sensitivity and specificity of LFDs in a systematic review and summarise the sensitivity and specificity of these tests. METHODS: A targeted search of Pubmed and Medxriv, using PRISMA principles, was conducted identifying clinical studies assessing the sensitivity and specificity of LFDs as their primary outcome compared to reverse transcriptase polymerase chain reaction (RT-PCR) for the detection of SARS-CoV-2. Based on extracted data sensitivity and specificity was calculated for each study. Data was pooled based on manufacturer of LFD and split based on operator (self-swab or by trained professional) and sensitivity and specificity data were calculated. RESULTS: Twenty-four papers were identified involving over 26,000 test results. Sensitivity from individual studies ranged from 37.7% (95% CI 30.6–45.5) to 99.2% (95% CI 95.5–99.9) and specificity from 92.4% (95% CI 87.5–95.5) to 100.0% (95% CI 99.7–100.0). Operation of the test by a trained professional or by the test subject with self-swabbing produced comparable results. CONCLUSIONS: This systematic review identified that the performance of lateral flow devices is heterogeneous and dependent on the manufacturer. Some perform with high specificity but a great range of sensitivities were shown (38.32–99.19%). Test performance does not appear dependent on the operator. Potentially, LFDs could support the scaling up of mass testing to aid track and trace methodology and break the chain of transmission of COVID-19 with the additional benefit of providing individuals with the results in a much shorter time frame. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12879-021-06528-3.

Lateral flow device (LFD) immunoassays are common, inexpensive, readily available testing devices that are used in the detection of a number of different medical conditions [1] [2] [3] [4] . They work by binding of conjugated antibodies to a specific antigen in a sample. This antibody-antigen complex moves via capillary flow to a test area which then identifies a positive test by the presence of a coloured line [2, 3] .

There has been an increasing number of papers reporting on the use of LFDs in the detection of the Severe Acute Respiratory Syndrome Coronavirus 2 Open Access *Correspondence: dylan.mistry@nhs.net 1 Oxford University Hospitals, Headley Way, Headington, Oxford OX3 9DU, UK Full list of author information is available at the end of the article (SARS-CoV-2), which has caused the Coronavirus disease 2019 (COVID-19) pandemic [5] . Currently, the gold standard for detection of SARS-CoV-2 is reverse transcriptase polymerase chain reaction (RT-PCR) [6, 7] . For both of these tests, nasopharyngeal swabs are used to isolate the antigen. However, RT-PCR requires swabs to be sent off to a laboratory with specialist equipment and analysed by trained laboratory staff. This usually has a turnaround time that is variable but of at least 24 h [1, 7] . Furthermore, many countries possess a limited capacity to perform RT-PCR tests, hindering their ability to engage in mass-testing with RT-PCR alone; as an example, the United Kingdom's current RT-PCR capacity for the detection of SARS-CoV-2 is approximately 500,000 tests per day [8] . Where there are national or local outbreaks, it is important to be able to expand testing in a short time frame (surge-testing) to enable effective identification of individuals infected with the virus for contact tracing and mass population testing in an endeavour to stop the chain of transmission of the virus [5, 9] . Lateral flow devices (LFDs) offer a potential solution as they can quickly turn around a result in less than 30 min without the need for specialist staff or laboratory capacity [2, 3] . Many countries have pioneered the use of LFDs for surge-testing in the healthcare, community and educational setting [10, 11] .

To date, there has yet to be a systematic review to assess the sensitivity and specificity of LFDs in the detection of SARS-CoV-2 without which a thorough evaluation of the efficacy of these tests cannot be undertaken.

The primary objective was to identify the sensitivities and specificities of lateral flow devices in the detection of SARS-CoV-2 compared to reverse transcriptase polymerase chain reaction in patients with symptoms of COVID-19 or those screened as part of mass testing programmes. This study also set out to identify if there were any differences in sensitivity and specificity between different manufacturers of LFDs and between different operators of the LFD test.

This was a systematic review of clinical studies in peer reviewed journal articles.

Two independent reviewers conducted an electronic search strategy of two online databases, PubMed and Medxriv, in 1st December 2020 to 15th January 2021. Search terms used included but not exclusively a combination of "COVID-19", "SARS-CoV-2", "CORONAVI-RUS", "ANTIGEN DETECTION", "ANTIGEN TEST", "LATERAL FLOW". The two reviewers then reviewed each paper generated from the search and excluded articles based firstly on title then abstract and then reviewing the full text. References of the filtered papers were searched for additional studies. Any disagreements between the reviewers were resolved by consulting a separate adjudicator and a discussion between all three parties.

Eligible studies had to meet the following criteria: (1) involved the detection of SARS-CoV-2, (2) the intervention was a LFD detecting the antigen to this virus, (3) the LFD was performed at the point of care on samples taken for this purpose, (4) the control used as the "gold standard" must be RT-PCR, (5) outcomes for the paper must include the sensitivity and specificity of the lateral flow device, (6) population must be adults (≥ 18 years) who displayed symptoms of COVID-19 or swabbed as part of screening or mass testing, (7) the full text must be published in peer reviewed journals or a preprint pending review at the time of the search.

Exclusion criteria included any study that did not meet all the conditions for eligibility and: (1) was detecting anything other than SARS-CoV-2, (2) retrospectively tested samples which had been frozen, (3) tested exclusively healthy volunteers with no indication for swabbing, (4) did not provide appropriate sensitivity and specificity data.

Once all papers from the search had been identified the two independent reviewers reviewed the full text of all identified papers. Descriptive data for each article were identified including author, month and year, location, sample size and manufacturer of LFD used. The reviewers then extracted test result data including the number of participants in which SARS-CoV-2 was detected by RT-PCR and LFD and the number of false positive and negative results detected by LFDs. Sensitivity and specificity data were collected for each study including 95% confidence intervals; in all studies, this was calculated to confirm the sensitivity and specificity data. The data was subsequently split and pooled based on the manufacturer of LFD used which enabled calculation of sensitivity and specificity for each manufacturer of LFD compared to RT-PCR. Studies were split again if the sample was taken by a trained professional or if it was taken by the patient with self-swabbing, regardless of who operated the LFD test. Sensitivity and specificity data were calculated comparing these two groups. Again, any disagreements during data extraction were settled by consulting the third party. 

The pre-defined primary outcome was to assess the sensitivity and specificity of LFD tests in the detection of SARS-CoV-2 compared to RT-PCR ("gold standard") testing in patients with symptoms consistent with COVID-19 or in individuals swabbed as part of mass population testing/contact tracing. The secondary outcome was to calculate the sensitivity and specificity of bar chart (B) . A In the "other" group in Abdelrazik et al. refers to exposed healthcare professionals (close contacts were a separate group in this trial too). For Cerutti et al., this refers to patients who were tested from "high risk" travel areas as deemed by the local government each LFD test by manufacturer in this same population in comparison to RT-PCR and based upon whether the sample collection was performed by a trained professional or by the patient ("self-swabbing").

Data analysis was conducted using IBM SPSS Version 27.0.0. For the primary outcome in the majority of studies, no data analysis was required as all results were extracted from articles directly. For the secondary outcome, results of individual manufacturers of LFDs were pooled together and a sensitivity/specificity analysis conducted. A total sensitivity and specificity were reported for each manufacturer with 95% confidence intervals. Data visualisation was performed in R version 4.0.3. Heatmaps and Forest plots were generated using the pheatmap() function of the 'pheatmap' (v1.0.12) and forestplot() function of the 'forestplot' (v1.10.1) R packages, respectively. Bar plots, horizontal dot plots and pie charts were generated using the geom_bar(), geom_line(), geom_point() and coord_polar() functions of the 'ggplot2' (v3.3.2) R package, respectively.

The search strategy yielded 1345 papers and further titles were identified by checking the references of these articles. This was narrowed down to 24 full text articles as demonstrated by the PRISMA flow diagram from in Fig. 1 . In total 26,903 tests were included in these 24 articles, which are summarised in Table 1 , including sample sizes, population and LFD type used. There was an almost equal gender split and a range of different test centres such as COVID-19 test centres and primary care centres ( Fig. 2 and Additional file 1: Appendix 1).

The indication for testing for SARS-CoV-2 of the participants [e.g., screening or (a)symptomatic testing, close contacts] are included in Fig. 3 , demonstrating that the systemic review contains a diverse population sample that would be representative of those being tested for COVID-19.

Eight different manufacturers of LFDs were used across 24 studies. Panbio Abbot had the highest number of publications and was used across 12 different studies with a combined total of 13,000 tests. This is demonstrated in Fig. 4 and Additional file 1: Appendix 2.

Individual study sensitivity and specificity data is demonstrated by Table 2 . This shows a range of sensitivity from 37.7% (95% CI 30.6-45.5) from Blairon et al. [16] (which used the CORIS LFD) to Moeren et al. [29] with a sensitivity of 99.2% (95% CI 95.5-99.9) using the BD Veritor LFD test, as demonstrated by Fig. 5A. For specificity, all studies demonstrated a specificity over 92%. Eleven studies had a specificity of 100%. This is demonstrated in Fig. 5B .

After combining studies based on manufacturer of LFD, BD Veritor had the best sensitivity of 99.19% (95% CI 95.54-99.86%), though the sample size was small and it Table 2 Sensitivity and specificity data extracted from each study *Shows data which had slight variations between our data calculations and the calculations made in the study, possibly due to a different method for calculating 95% confidence intervals. † Shows data that produced significant differences in between our calculated data and the study's data or it was not possible to calculate sensitivity and specificity from the data in the study was only tested from a single centre study. The CORIS and BIOSENSOR were the lowest sensitivity LFDs demonstrating sensitivities of less than 45%. Panbio Abbott has been most thoroughly evaluated and noted a sensitivity of 78.41% (95% CI 76.78-79.96%) across over 2500 individual tests. All manufacturers demonstrated a specificity of over 93% and three (BD Veritor, BIOCREDIT, COVID-VIRO) had specificities of 100%. This is shown in Table 3 and Fig. 6 .

Studies were split by sample collector as displayed in 

This systematic review has identified, across 24 studies and over 26,000 LFD tests, that a number of individual manufacturers of LFDs recorded a sensitivity of over 78% compared to the gold standard test of RT-PCR, with one individual manufacturer reaching up to 99.19% sensitivity in one single centred trial (BD Veritor). Specificity was more consistent, with over 92% in all individual studies and from the pooled data. The large variation between brands of LFDs could be due to several factors including individual study design, operator competencies but also quality of the LFD itself. This highlights the impressive performance of the Panbio Abbot and Innova brands both with sensitivities of over 78% but with a sample size of 13,221 and 6954 respectively. This study is the first to summarise the existing body of studies to help create a broader understanding for LFD testing for SARS-CoV-2 and is the first systematic review of its kind. While RT-PCR is and is likely to remain the gold standard of testing, this study highlights the potential utility of rapid antigen testing to support RT-PCR in the scaling up of a country's testing program to include mass testing, contact tracing programs and potentially surge-testing [9, 36] . Potential use of LFDs might be to provide short term additional capacity, or as an adjunct to PCR testing [1, 7, 8] . The lower sensitivity demonstrated by certain brands of LFDs compared to RT-PCR can be overcome to an extent in high prevalence areas with appropriate frequency of testing. LFDs may come into their own when used in areas with big spikes in cases. We note that there is an increasing body of modelling data highlighting that the best surveillance testing methods are tests that can be scaled up and reported quickly, [36] requirements which LFDs may have suitable characteristics. These models also highlight the need for recurrent testing. This again is a requirement LFDs can fulfil given their minimal expense. High frequency testing in high prevalence areas may negate some concerns around sensitivity [36] . In contrast, low incidence areas A B C Fig. 7 The proportions of LFD tests by sample collector is displayed in A. The sensitivity of LFD tests by sample collector with 95% confidence intervals is displayed as a Forest Plot in B. The specificity of LFD tests by sample collector with 95% confidence intervals is displayed as a Forest Plot in C would expose the inferior sensitivities demonstrated by LFDs in this study, and RT-PCR would be the most suitable, especially if there is a reduction in demand for mass population and high frequency testing in these areas. This point highlights that whilst LFDs have some benefits, when compared directly to RT-PCR, their performance when detecting SARS-CoV-2 was inferior and as such they should be utilised when RT-PCR is overwhelmed.

Our study design is not without its limitations. There are possible confounding variables including the marked heterogeneity in terms of study designs whereby some targeted asymptomatic or symptomatic groups, and others targeted contacts of symptomatic patients. However, as there was a variety of settings and scenarios to replicate the conditions of real-life testing, this data can still provide valuable insight into the performance of LFDs.

Furthermore, this systematic review takes the assumption that for the diagnosis of COVID-19, RT-PCR testing is the most appropriate measure for comparison. There is a debate whether RT-PCR testing is the most appropriate method in a high-incidence setting [37] . In such a setting RT-PCR might actually report an overall greater number of positive cases than those which should be considered active infections, because of the presence of residual RNA which can be present for several months after an initial infection with SARS-CoV-2 [37] [38] [39] . Other measures of assessing the infectivity of individuals, such as viral culture, might provide better measurements but suffer from other logistical implementation issues.

On a final note, caution should be exerted particularly in view of new emergent strains. The sensitivity of any COVID-19 tests to new strains, not least LFDs must be confirmed. Several such evaluations have been completed by Public Health authorities in the United Kingdom and have given reassurance in this regards [40] .

In summary, this systematic review has shown that lateral flow devices can produce varying sensitivity and specificity results compared to the other forms of SARS-CoV-2 diagnostics. We have shown that a number of manufacturers of LFDs can produce high specificity but there is significant heterogeneity in sensitivity (38.32-99.19%) , which may suit LFD use to high prevalence areas in an attempt to rapidly increase testing in areas with raised transmission. Our evidence gives support to the practice of self-swabbing for sample collection compared to the test being performed by a trained healthcare professional. LFDs potentially offer a new form of COVID-19 testing that might ease the pressure on the RT-PCR testing program. Enhanced capacity for mass testing, contact tracing and surge-testing, may in turn help stop the chain of transmission of COVID-19.

Report from the American society for microbiology COVID-19 international summit

Evolution in lateral flow-based immunoassay systems

Fast coronavirus tests: what they can and can't do

Evaluation of a novel antigen-based rapid detection test for the diagnosis of SARS-CoV-2 in respiratory samples

Laboratory testing strategy recommendations for COVID-19

How is the COVID-19 virus detected using real time RT-PCR?

Emergency Use Authorisation (EUA) Summary COVID-19 RT-PCR Test

The United Kingdom Government. UK Daily Coronavirus Summary

Covid-19 mass testing programmes

Covid-19: mass testing in Slovakia may have helped cut infections

More rapid COVID-19 tests to be rolled out across England

Potential use of antigen-based rapid test for SARS-CoV-2 in respiratory specimens in low-resource settings in Egypt for symptomatic patients and high-risk contacts

Comparison of SARS-COV-2 nasal antigen test to nasopharyngeal RT-PCR in mildly symptomatic patients

Field evaluation of a rapid antigen test (Panbio ™ COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres

Diagnostic accuracy of two commercial SARS-CoV-2 Antigen-detecting rapid tests at the point of care in communitybased testing centers

Implementation of rapid SARS-CoV-2 antigenic testing in a laboratory without access to molecular methods: experiences of a general hospital

Evaluation of the Panbio ™ rapid antigen test for SARS-CoV-2 in primary health care centers and test sites

Urgent need of rapid tests for SARS CoV-2 antigen detection: evaluation of the SD-Biosensor antigen test for SARS-CoV-2

Rapid SARS-CoV-2 antigen detection assay in comparison with real-time RT-PCR assay for laboratory diagnosis of COVID-19 in Thailand

Real-life performance of a novel antigen detection test on nasopharyngeal specimens for SARS-CoV-2 infection diagnosis: a prospective study

The sensitivity of SARS-CoV-2 antigen tests in the view of large-scale testing

Real-life validation of the Panbio COVID-19 antigen rapid test (Abbott) in community-dwelling subjects with symptoms of potential SARS-CoV-2 infection

Clinical evaluation of the Roche/SD Biosensor rapid antigen test with symptomatic, non-hospitalized patients in a municipal health service drive-through testing site

Evaluation of the accuracy and ease-of-use of Abbott PanBio-a WHO emergency use listed, rapid, antigen-detecting point-of-care diagnostic test for SARS-CoV-2

Evaluation of the accuracy, ease of use and limit of detection of novel, rapid, antigen-detecting point-of-care diagnostics for SARS-CoV-2

Panbio antigen rapid test is reliable to diagnose SARS-CoV-2 infection in the first 7 days after the onset of symptoms

Nasopharyngeal Panbio COVID-19 antigen performed at point-of-care has a high sensitivity in symptomatic and asymptomatic patients with higher risk for transmission and older age

Multicenter evaluation of the Panbio ™ COVID-19 rapid antigen-detection test for the diagnosis of SARS-CoV-2 infection

Performance evaluation of a sars-cov-2 rapid antigentest: test performance in the community in the Netherlands

Field evaluation of the performance of a SARS-CoV-2 antigen rapid diagnostic test in Uganda using nasopharyngeal samples

COVID-19: Rapid Antigen detection for SARS-CoV-2 by lateral flow assay: a national systematic evaluation for mass-testing

PMC7263236; evaluation of a novel antigen-based rapid detection test for the diagnosis of SARS-CoV-2 in respiratory samples

Antigen rapid tests, nasopharyngeal PCR and saliva PCR to detect SARS-CoV-2: a prospective comparative clinical trial

Real-life evaluation of a rapid antigen test (Panbio ™ COVID-19 Ag Rapid Test Device) for SARS-CoV-2 detection in asymptomatic close contacts of COVID-19 patients

Diagnosis value of SARS-CoV-2 antigen/antibody combined testing using rapid diagnostic tests at hospital admission

Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance

False-Negative Results Of Initial RT-PCR assays for COVID-19: a systematic review

Positive RT-PCR test results in patients recovered from COVID-19

Covid-19: UK test and trace system still missing 80% target for reaching contacts

Rapid evaluation confirms lateral flow devices effective in detecting new COVID-19 variant

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

The authors would like to thank the authors of the 24 studies used in this systematic review for their contribution to the collection research in the fight against COVID-19. They would like to thank all the doctors, nurses and other clinical staff working on the frontline of healthcare authorities worldwide and those who have suffered or are suffering from COVID-19.

LFD: Lateral flow device; RT-PCR: Reverse transcriptase polymerase chain reaction.

The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s12879-021-06528-3.Additional file 1. Appendix 1 -Gender split for each paper included in the study. Appendix 2 -Sample size based on manufacturer of LFD used. 

No funding was obtained for this study.

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Ethics approval and consent to participate Not applicable.

Not applicable.

The authors declare that they have no competing interests.