key: cord-0983424-hwolws7y
authors: David, Anura; Scott, Lesley; Jugwanth, Sarika; Gededzha, Maemu; Kahamba, Trish; Zwane, Nontobeko; Mampeule, Nakampe; Sanne, Ian; Stevens, Wendy; Mayne, Elizabeth S.
title: Operational characteristics of 30 lateral flow immunoassays used to identify COVID-19 immune response
date: 2021-07-06
journal: J Immunol Methods
DOI: 10.1016/j.jim.2021.113096
sha: d2751fb2b9ce804e7e79073ea689279ba0aa7800
doc_id: 983424
cord_uid: hwolws7y

Serology or antibody tests for COVID-19 are designed to detect antibodies (mainly Immunoglobulin M (IgM) and Immunoglobulin G (IgG) produced in response to Severe Acute Respiratory Syndrome Coronavirus 2 (SARS CoV-2) infection. In this study, 30 lateral flow immunoassays were tested using serum or plasma from patients with confirmed SARS CoV-2 infection. Negative serological controls were accessed from a well-characterised bank of sera which were stored prior to February 2020. Operational characteristics and ease of use of the assays are reported. 4/30 (13%) of kits (Zheihang Orient Gene COVID-19 IgG/IgM, Genrui Novel Coronavirus (2019-nCoV) IgG/IgM, Biosynex COVID-19 BSS, Boson Biotech 2019-nCoV IgG/IgM) were recommended for SAHPRA approval based on kit sensitivity. Of these, only the Orientgene was recommended by SAHPRA in August 2020 for use within the approved national testing algorithm while the remaining three received limited authorization for evaluation. All kits evaluated work on the same basic principle of immunochromatography with minor differences noted in the shape and colour of cartridges, the amount of specimen volume required and the test duration. Performance of the lateral flow tests were similar to sensitivities and specificities reported in other studies. The cassettes of the majority of kits evaluated (90%) detected both IgG and IgM. Only 23% of kits evaluated contained all consumables required for point-of-care testing. The study highlights the need for thorough investigation of kits prior to implementation.

Lateral flow immunoassays are paper-based platforms where a liquid specimen (typically whole blood, serum or plasma in the case of serology) containing the analyte of interest moves through capillary action through various zones of polymeric strips, on which molecules that can interact with the analyte are attached [11] . If the analyte of interest is present in the specimen, the presence of a test line is seen within 5-30 min. LFIAs require minimal laboratory space and equipment, can be easily interpreted, and can be performed with minimal training by testing personnel. This makes these assays attractive for field testing particularly in remote sites with limited laboratory access. A number of these tests have been marketed for SARS CoV-2 although there is substantial variability between different tests including the contents of the kits, specimen volume required, and other operational characteristics such as the ease of use, the test duration and the specimen type required. Rapid SARS-CoV2 tests have been developed across the world but have also shown markedly variable performance resulting in recalls in some regulatory jurisdictions including the Food and Drug Administration of the United States [12] .

In South Africa, the National Health Laboratory Service (NHLS) is the primary supplier of pathology services to the state healthcare sector (80% of the population). It also serves a reference and advisory service to the South African Health Products Regulatory Authority (SAHPRA) in the test licensing process.

It is, therefore, responsible for assessing test performance of assays for implementation in South Africa including those assays which can be utilized for near or at-patient care. The Immunology Laboratory at NHLS Braamfontein is an ISO15189:2012 accredited facility with expertise in infectious disease serology testing. In collaboration with Clinical Laboratory Services (Department of Molecular Medicine and Haematology, Johannesburg, South Africa) [CLS], the laboratory undertook to evaluate all LFIAs, that received SAHPRA-approval for import, in order to assist the regulatory authority with licensing decisions and also to investigate the clinical utility and ease of use of these assays. 

This study was approved by the Human Research Ethics Committee of the University of the Witwatersrand (M200468). Participants who tested positive by RT-PCR for at least 2 genes were requested to participate in the study and then enrolled, following informed consent (n=587). Bloods were collected at a single time-point for each participant. An average of 150 samples were tested post-PCR and care was taken to ensure that representative samples of a pre-defined panel was used for each kit evaluated. Clinical staff collected study specimens while adhering to standard health and safety precautions required when handling confirmed and suspected cases of COVID-19 [12, 13] . Participants were requested to provide 2 X 6mL Ethylenediamine tetraacetic acid (EDTA) tubes of blood, 4 X 4mL serum separator tubes for serum storage, a nasal swab and saliva and to complete a questionnaire containing information on symptoms. Upon receipt at the testing laboratory (CLS) biorepository, specimens were immediately processed, sub-aliquoted and stored frozen at -80°C. Freeze-thaw cycles were limited to one. The median time between sampling and onset of symptoms was 35 days (range: 10 -60 days).

Patients infected with the SARS CoV-2 virus were recruited from i) clinical facilities admitting and isolating infected individuals, ii) volunteers at various clinical centres and iii) among those tested at various NHLS facilities in Gauteng and the Western Cape. Patients were stratified according to the number of days post-symptom or post-PCR diagnosis.

autoimmune diseases with high polyspecific antibody production including rheumatoid arthritis and systemic lupus erythematosus and serum samples from patients with pneumonia of viral aetiology prior to 2020.

In this evaluation, two independent reference standards were utilized. PCR testing as conducted for initial diagnosis was utilized as an initial standard. PCR positivity was determined by RT-PCR laboratory testing and confirmed through an independent database. Where possible, the PCR test was repeated to confirm the initial result. A serological ELISA-based assay (EUROIMMUN Medizinische Labordiagnostika AG, Germany) was utilized as a secondary serological reference methodology. The EUROIMMUN assays were two of the first antibody detection tests to be validated and available for COVID-19 testing and uses the recombinant S1 domain of the SARS-CoV-2 spike protein as antigen. Formal serology was provided for reference purposes and did not form a component of the analysis for performance.

LFIAs were recommended for evaluation after review of a technical dossier by SAHPRA. The majority of kits were donated for evaluation purposes.

All LFIAs were evaluated using a standardized protocol in 2 phases. An initial evaluation was undertaken utilizing 25 positive controls and 25 negative controls. For tests which met the acceptance criteria (defined as a sensitivity ≥85% and a specificity ≥98%) for IgG detection, a secondary evaluation testing an additional 100 specimens was undertaken.

Evaluation procedure J o u r n a l P r e -p r o o f

The LFIA cassettes use whole blood (venous and/or fingerprick), serum or plasma to detect the presence of antibodies (IgG or IgM) produced against a specific SARS-CoV-2 antigen. Examples of the some of the LFIAs evaluated are shown in Figure 1 . All lateral flow cassettes evaluated contained a conjugate pad with recombinant COVID-19 antigens conjugated to colloidal gold (COVID-19 conjugates) and IgG and IgM-colloidal gold conjugates. Once diluted serum, plasma or whole blood is applied to the conjugate pad section, IgM and/or IgG antibodies, if present, will bind to COVID-19 conjugates to form an antigenantibody complex. The complex then moves across the nitrocellulose membrane by capillary action.

When the complex meets the line of the corresponding immobilized antibody (anti-human IgM and/or anti-human IgG), it is trapped to form a coloured band, which indicates a positive test result. The absence of a coloured band in the test region indicates a negative test result. The test contains an internal control band (C band) which should produce a coloured band for each test. This control line should be visible after the detection step as this confirms that the kit is working as intended. For the purpose of evaluation, plasma or serum was used and testing was performed as per package insert instructions. Minimal training was required for the testing procedure. Briefly, tests were performed by applying the serum or plasma specimen to the sample well of the cassette followed by the addition of specimen diluent. For the purposes of uniformity, a micropipette was used to ensure that the correct volume of specimen was added to each well. At the time of application, a timer was started and the strip was observed for the formation of control and test lines, following the incubation period recommended by the manufacturer. performance for each of the 30 kits was measured by assessing the sensitivity, specificity and/or kappa co-efficient of the IgG LFIA result against PCR and formal serology. In addition to analytical performance, J o u r n a l P r e -p r o o f a desirable and acceptable target product profile was identified which comprised shipping and storage temperature, whether the kit detects IgG and/or IgM, specimen type required, recommended setting for use (laboratory or primary healthcare facility), test duration, kit contents and ease of use. To ensure that all kits were evaluated in a fair manner, all testing was performed using a standardized protocol, specimen type and panel. To determine performance, the testing panel used fulfilled World Health Organisation (WHO) criteria to include majority (96%) of patients >21 days post onset of symptoms and also included specimens with different days of onset since illness. IgM was not used as an evaluation criteria since tests were conducted on specimens from individuals with a median of 35 days postsymptom onset when IgM levels were expected to be reduced, the formal comparative assays did not detect IgM and IgM is not produced by a proportion of infected individuals [13] .

Only 4% of specimens tested were representative of the <14 day post-PCR category, hence data was not aggregated for performance analysis. 11% of participants tested were asymptomatic. For IgG, sensitivity of the kits in comparison to PCR and formal serology, ranged from 17% (95% CI: 4.74-37.4) to 96% (95% CI: 79.6-99.9) and 17% (95% CI: 4.74-37.4) to 100% (95% CI: 94.5-100), respectively with specificities ranging from 72% (95% CI: 50.6-87.9) to 100% (95% CI: 95.2-100) and 63% (95% CI: 44.9-78.5) to 100% (95% CI: 90.5-100), respectively (Table 1) were similar in design, with minor differences in the shape, colour and size of cassettes. Other differences noted is that the labelling of test regions (IgM, IgG and C) on some cassettes were a different colour to the cassette and hence easier to read than others where the colours were the same. One kit contained cassettes in a bright blue colour which clearly distinguished this cassette from the others.

Lateral flow or point-of-care serology tests represent a useful analytical tool for SARS-CoV2 detection.

This is particularly true in African settings where clinics may be remote from laboratories. There has been, however, inconsistent performance resulting in kit recalls in many centers [12] . This necessitates thorough evaluation prior to selection implementation of any specific product. Although multiple reviews of LFIA performance have been published to date [9, 14, 15] For kits where a small volume (2 µl -5 µl) of specimen is required, the package inserts recommends the use of a micropipette to ensure that the correct volume of specimen is added to the cassette. This presents a concern for implementation of this kit in the field where micropipettes are not readily available especially at point-of-care. An additional point to consider is that, specimen droppers provided with the majority of kits were often difficult to use which could have cost implications if specimen droppers need to be purchased separately.

Time to positivity for the different kits showed no relation with volume of specimen required or length of incubation and all kits had acceptable incubation periods for a rapid point-of-care test. Result interpretation was difficult for some kits where cassettes produced faint bands. False positive results are a concern for these kits since the package inserts state that "any shade of colour in the test region J o u r n a l P r e -p r o o f should be considered positive." Some contained separate wells for the addition of specimen and diluent while others contained a single well for the addition of both.

Almost all kits detected IgM and IgG on the same cassette. The kit that offered separate cassettes for IgM and IgG testing had the following disadvantages compared to those where just one cassette was required: a larger volume of serum/plasma was required for testing and two tests required interpretation which translates to additional labour and time. The kit that detected IgG only has the obvious disadvantage that detection of IgM will be missed, if present in the patient specimen. Individual specimen ampules decreased risk of contamination however, the ampules were difficult to open, even with scissors and the diluent leaked out, causing spillage and a change of gloves between each test. A bulk specimen diluent contained in a single vial, however, has the disadvantage of cross-contamination and easily being misplaced.

Use of kits where testing can be performed on venous blood only requires a phlebotomist which will make implementation in a point-of-care setting difficult if fingerstick blood cannot be used. The fact that testing need to be performed within 30-60 minutes once the cassette is removed from its pouch needs to be taken into consideration for implementation at point-of-care sites where practice (for other kit types) often entails the unpacking of kits in advance to allow for easier use, when required. Of the 30

LFIAs evaluated, four were provided the status of SAHPRA approval (full or conditional) yet only one can be used as a standalone kit where required consumables will not need to be procured at an additional expense.

WHO Declares COVID-19 a Pandemic

Coronaviruses: an overview of their replication and pathogenesis

Coronaviruses and SARS-CoV-2: A Brief Overview. Anesthesia and analgesia

Serological assays for emerging coronaviruses: challenges and pitfalls

Patterns of IgG and IgM antibody response in COVID-19 patients. Emerging Microbes & Infections, 2020

CoV-2 diagnostic pipeline

Diagnostic performance of seven rapid IgG/IgM antibody tests and the

Euroimmun IgA/IgG ELISA in COVID-19 patients

Four point-of-care lateral flow immunoassays for diagnosis of COVID-19 and for assessing dynamics of antibody responses to SARS-CoV-2

Diagnostic accuracy of serological tests for covid-19: systematic review and meta-analysis

COVID-19 Serological Tests: How Well Do They Actually Perform? Diagnostics

Lateral flow assays

Removal Lists of Tests that Should No Longer Be Used and/or Distributed for COVID-19: FAQs on Testing for SARS-CoV-2

Non-infectious status indicated by detectable IgG antibody to SARS-CoV-2

Diagnostic performance of COVID-19 serology assays

Molecular and Serological Tests for COVID-19. A Comparative Review of SARS-CoV-2 Coronavirus Laboratory and Point-of-Care Diagnostics

An evaluation of COVID-19 serological assays informs future diagnostics and exposure assessment

VSARS-CoV-2 infection serology: a useful tool to overcome lockdown? Cell Death Discovery

Antibody testing for COVID-19: A report from the National COVID Scientific Advisory Panel. 2020. 19. What is Kappa coefficient (Cohen's Kappa)