key: cord-0762377-esxudd66
authors: Xie, Xuping; Nielsen, Marisa C.; Muruato, Antonio E.; Fontes-Garfias, Camila R.; Ren, Ping
title: Evaluation of a SARS-CoV-2 Lateral Flow Assay Using the Plaque Reduction Neutralization Test
date: 2020-10-16
journal: Diagn Microbiol Infect Dis
DOI: 10.1016/j.diagmicrobio.2020.115248
sha: 562a4f501f4f4e8e8e31dcde89f478d66bb02805
doc_id: 762377
cord_uid: esxudd66

As new tests and technologies advance our understanding and diagnostic capabilities of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the coronavirus disease 2019 (COVID-19), they must be appropriately validated to make sure test performance is following manufacturer claims. In this study, we evaluated the Vazyme 2019-nCoV IgG/IgM Detection Kit, which is a lateral flow assay (LFA), by the plaque reduction neutralization test (PRNT) using 100 patient plasma/serum samples. As compared to the PRNT results, the Vazyme LFA had 95.9% sensitivity and 96.1% specificity. Along with the increased need for rapid, effective, and affordable point of care tests to help provide meaningful epidemiological data, we demonstrated that the Vazyme LFA performed well on IgG detection but cannot be judged on the performance of IgM detection using PRNT alone. However, our observation of the low IgM-positive rate supported the poor performance of IgM detection of this LFA which led to the disapproval of its Emergency Use Authorization (EUA) recently.

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), an emerging virus, has caused a global pandemic of coronavirus disease 2019 (COVID-19). Real-time, reverse transcriptase-polymerase chain reaction (RT-PCR) detection of SARS-CoV-2 nucleic acid in clinical specimens is the current diagnostic standard (1) . While PCR is useful for the diagnosis of active infection, it lacks information regarding possible SARS-CoV-2 infection prior to testing.

To address this need, various types of serologic tests have been developed to detect SARS-CoV-2 immunoglobulin (Ig)G and IgM antibodies, indicating prior infection (2, 3) . Serological testing has proven to be an important component of the overall estimate of SARS-CoV-2 disease incidence and prevalence (2) . These tests can help determine whether a person has been infected with SARS-CoV-2 and may help determine immune-status. While antibody tests do not diagnose active SARS-CoV-2 infection, they provide more realistic epidemiological information. In turn, this can help with vaccine evaluation, public health planning, and quarantine strategies (4) .

In response to this need, many SARS-CoV-2 lateral flow, point-of-care antibody tests have been developed (2) . Lateral flow assays (LFAs) are a rapid, cost-effective, and easy-to-use solution for facilities without the resources necessary for advanced platforms (3). However, due to the emergent nature of the pandemic, many of these tests have not been thoroughly validated nor have they been approved by regulatory agencies (4) . This has resulted in the distribution of tests that may or may not function according to the manufacturers' claims and performance assessments are limited (4).

The plaque reduction neutralization test (PRNT) is the current gold standard for SARS-CoV-2 serology tests (4) . Even so, most of the published literature evaluating SARS-CoV-2 LFAs, do not use PRNT as the comparison method (2, 3, 5) due to its low throughput and biosafety level III restrictions. In this study, we evaluated 2019-nCoV IgG/IgM Detection Kit (Colloidal Cold-Based) (Nanjing Vazyme Medical Technology Co., LTD, China) for its sensitivity, specificity, and cross-reactivity using a PRNT assay. During the early stage of COVID-19 pandemic, we searched for a reliable LFA to support a serosurveillance study. At that time, the Vazyme test was among a few LFAs under application for the Federal Drug Administration (FDA) Emergency Use Authorization (EUA). We, therefore, evaluated the Vazyme LFA test using the PRNT assay. After completing our evaluation, we found that the FDA did not approve the EUA for this test because of its poor IgM performance. Here we report our independent evaluation results of the LFA test.

Serum/plasma samples (100) used in this study were either from patients with SARS-CoV-2 RT-PCR tests done or from collections before the COVID-19 pandemic. For assay crossreaction evaluation, serum/plasma samples that were positive for other infections, antibodies, or vaccinations were used (Table 1) . To assess interference, serum samples containing a certain amount of albumin or elevated cholesterol/rheumatoid factors were included as well ( Table 1 ).

The research protocols regarding the use of human serum/plasma specimens were reviewed and approved by the University of Texas Medical Branch (UTMB) Institutional Review Board (IRB).

A standard double-layer PRNT was performed with each serum/plasma sample (6).

Specifically, serial dilutions of serum/plasma samples (1:20) for the first dilution followed by serial 1:2 dilutions) were mixed with an equal amount of wild-type SARS-CoV-2 virus suspension containing 100 plaque-forming units (PFU) in 100 µl. After incubating the mixtures at 37C for 1 hr, each virus-serum/plasma mixture was inoculated onto one well of a 6-well tissue culture plate containing a confluent monolayer of Vero E6 cells. After incubating the plate at 37°C for 1 h, an agar overlay was added to the infected cell monolayer, and the plate was further incubated at 37°C for 2 days. When virus plaques became visible, a second overlay containing neutral red was added and incubated for 5 hrs. Then neutral red was removed and plaques were counted. The antibody titer was determined as the serum/plasma dilution that inhibited 50% of the tested virus inoculum (PRNT 50 ).

The assay was done following the manufacture's instruction. Briefly, the test kits were warmed to room temperature, removed from the foil pouch, and placed horizontally on a flat surface. Using the provided dropper, 1 drop (~20 ul) of serum/plasma and 3 drops (~60 ul) of dilution buffer were added to the sample loading position. After 10 minutes, the test results were read, analyzed, and photographed.

This assay is a modified PRNT using mNeonGreen reporter SARS-CoV-2 instead of the wild-type virus in the neutralization test (7) . Briefly, after infecting cells with the mixture of serum/plasma and reporter virus, the plates were incubated for 16 hours. Intracellular mNeonGreen fluorescence signals were measured using Cytation TM 7 (BioTek). The infections without serum/plasma treatment were used as non-treatment controls. mNeonGreen fluorescence signals from the non-treatment controls were set at 100%. mNeonGreen fluorescence signals from each serum/plasma-treated samples were normalized to those from the non-treatment controls. The neutralization titers were defined by the serum/plasma dilution fold at which suppressed 50% of the mNeonGreen fluorescence signals of the non-treatment control (mNG-NT 50 ).

Statistical analysis was performed using Medcalc stats (http://www.medcalc.org/calc/diagnostic_test.php). Any faint band was considered to be positive during this study. Although the package insert describes the whole blood as a valid specimen type to be used, the whole blood was not evaluated in this study.

As compared to the PRNT results, the Vazyme 2019-nCoV IgG/IgM Detection Kit had 95.9% sensitivity (95% CI: 86.0 -99.5%) and 96.1% specificity (95% CI: 86.5 -99.5%). Of the 49 positive Vazyme LFA tests, only 10 were positive for IgM antibodies and 9 out of those 10 samples were also positive for IgG. There were one sample each with IgM(+) IgG(-) and IgM(+) IgG(+) were negative by PRNT (Table 2) . Of the 24 samples collected before the SARS-CoV-2 pandemic, none were positive for antibodies by either detection method.

In the early stages of disease (days 0-5), the Vazyme LFA detected antibodies in only 60% of the samples. In the intermediate and later stages of disease (6+ days) the assay detected antibodies in 100% of the positive samples ( Table 3 ). The discrepant results are further detailed in Table 4 . A newly developed mNeonGreen Reporter SARS-CoV-2 Virus Neutralization Test (Table 4) . Table 1 lists the common antibodies and common interference substances encountered in human serum/plasma. A few specimens collected within one to six months of post PCR positive on the common respiratory viral pathogens were also included. Vazyme LFA did not produce any false-positive results except one specimen containing anti-Cytomegalovirus (CMV) IgG.

information about the true prevalence of the disease. We have demonstrated that the Vazyme LFA accurately detects SARS-CoV-2 antibodies with almost no evidence of cross-reactivity with other antibodies or common interfering substances except possible cross-reaction to CMV IgG (1 out of 5, 20%) in our cases. Architect SARS-CoV-2 IgG assay (Abbott, Abbott Park, IL), a chemiluminescent microparticle immunoassay, also reported 1 out 5 CMV IgG positive samples was tested to be SARS-CoV-2 IgG positive (8) . However, the target for the antibody in Architect SARS-CoV-2 IgG assay is SARS-CoV-2 virus nucleocapsid protein, while Vazyme LFA targets the spike protein. To our knowledge, this is the first report to evaluate a LFA using the gold standard serology reference method PRNT while previous studies compared LFA to enzymelinked immunosorbent assay (ELISA) (2, 5, 9) . The most significant advantage of PRNT is detecting and measuring antibodies capable of neutralizing a target virus, while ELISA and other immunoassays cannot determine this neutralization capability.

While Vazyme LFA detected SARS-CoV-2 IgG with high sensitivity and specificity it did not appear to perform well with IgM detection. Our observation is in agreement with the poor IgM performance of this LFA, which led to the disapproval of its EUA by the FDA (10).

Studies have shown that a positive IgM result can be detected as early as the fourth day after symptom onset of COVID-19 and last for at least a few weeks (9) . Therefore, IgM should be present in most of the positive serum/plasma samples used in this study since about 70% of the samples were collected from symptomatic patients that were 6-20 days post-positive SARS-CoV-2 nucleic acid detection. However, IgM was only detected in 9 samples (18%).

Unfortunately, PRNT only measures total antibodies and can not differentiate between IgG and IgM, unless one type of Ig is depleted from serum/plasma samples before performing the assay.

However, the depletion experiment was not performed in this study. Therefore, the sensitivity and the specificity of Vazyme LFA on IgM cannot be evaluated accurately. Despite our findings, other SARS-CoV-2 LFA has also shown lower sensitivity for IgM antibodies, and this may be due to the lack of strength in the initial IgM signal during the early stages of SARS-CoV-2 infection (11) . Since not all spike antibodies could neutralize virus infection, it is expected higher titers of antibodies could be detected by spike protein-based binding assay (e.g., LFA, ELISA, or Luminex) than those detected by PRNT. However, there should be a correlation between the titers detected by the spike protein-based binding assay and PRNT assay. Indeed, our results support such concordance. However, antibody response can be unpredictable and there is still much to learn about human responses to SARS-CoV-2 infection (11) .

Although RT-PCR detection of SARS-CoV-2 nucleic acid in clinical specimens is the current diagnostic standard, this technology requires specialized instrumentation and highly trained personnel. It is usually performed only at larger hospitals, reference, and public health laboratories (12) . Unfortunately, smaller hospitals, doctor's offices, and clinics may not perform complex testing on site. Many of these locations opt to send patients to larger hospitals for testing, often delaying patient care, treatment, and results. Rapid, effective, and affordable alternatives to RT-PCR and ELISA are necessary to help integrate decentralized testing and provide meaningful epidemiological data to public health officials during this unprecedented pandemic (11) . Therefore, it is important to appropriately assess new SARS-CoV-2 assays and many of the point-of-care LFAs that have not been adequately tested. To meet the diagnostic need of the COVID-19 pandemic, many companies and academic labs developed different diagnostic platforms before or when the regulatory agency has stipulated or refined the standards for diagnostic products. Thus, thorough evaluations of these products are essential to ensure the accuracy of these diagnostic products. Early guidance from the regulatory agency is critical to improving the development of diagnostic tests, particularly during a pandemic crisis. In this study, we have demonstrated that the Vazyme LFA accurately detected SARS-CoV-2 IgG but not necessarily IgM. Therefore this LFA might be a reliable point-of-care test for IgG detection but not for IgM in facilities without resources for complex testing. 1 0 Yellow fever virus post-immunization 2 0 $ The immune sera are listed in alphabetical order. Samples tested positive for antibodies against specific pathogens are indicated with the prefix "anti", whereas samples tested positive on antigens or pathogen nucleic acids are not indicated with the prefix. # Specimens with interfering substances ^Specimens collected within one to six months after PCR tested positive 

Laboratory diagnosis of emerging human coronavirus infections-the state of the art. Emerg Microbes Infect

Four point-of-care lateral flow immunoassays for diagnosis of COVID-19 and for assessing dynamics of antibody responses to SARS-CoV-2

Development and Clinical Application of A Rapid IgM-IgG Combined Antibody Test for SARS-CoV-2 Infection Diagnosis

Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Responses in Coronavirus Disease Patients

Evaluation of two automated and three rapid lateral flow immunoassays for the detection of anti-SARS-CoV-2 antibodies

A high-throughput neutralizing antibody assay for COVID-19 diagnosis and vaccine evaluation

An Infectious cDNA Clone of SARS-CoV-2

Abbott Architect SARS-CoV-2 IgG assay package insert

Evaluation of eleven rapid tests for detection of antibodies against SARS-CoV-2

Fast, portable tests come online to curb coronavirus pandemic

Quantification using real-time PCR technology: Applications and limitations

All authors have no conflict of interest.

The project is approved by the UTMB Institutional Review Board (IRB) under an expedited review process with IRB # 20-0070 & 20-0143.

Not applicable.