key: cord-0854562-5xvs4tpu
authors: Yamayoshi, Seiya; Sakai-Tagawa, Yuko; Koga, Michiko; Akasaka, Osamu; Nakachi, Ichiro; Koh, Hidefumi; Maeda, Kenji; Adachi, Eisuke; Saito, Makoto; Nagai, Hiroyuki; Ikeuchi, Kazuhiko; Ogura, Takayuki; Baba, Rie; Fujita, Kensuke; Fukui, Takahiro; Ito, Fumimaro; Hattori, Shin-ichiro; Yamamoto, Kei; Nakamoto, Takato; Furusawa, Yuri; Yasuhara, Atsuhiro; Ujie, Michiko; Yamada, Shinya; Ito, Mutsumi; Mitsuya, Hiroaki; Omagari, Norio; Yotsuyanagi, Hiroshi; Iwatsuki-Horimoto, Kiyoko; Imai, Masaki; Kawaoka, Yoshihiro
title: Comparison of Rapid Antigen Tests for COVID-19
date: 2020-12-10
journal: Viruses
DOI: 10.3390/v12121420
sha: 5a36c81182be11f4d2e1d55827f84e16714d5bb4
doc_id: 854562
cord_uid: 5xvs4tpu

Reverse transcription-quantitative PCR (RT-qPCR)-based tests are widely used to diagnose coronavirus disease 2019 (COVID-19). As a result that these tests cannot be done in local clinics where RT-qPCR testing capability is lacking, rapid antigen tests (RATs) for COVID-19 based on lateral flow immunoassays are used for rapid diagnosis. However, their sensitivity compared with each other and with RT-qPCR and infectious virus isolation has not been examined. Here, we compared the sensitivity among four RATs by using severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) isolates and several types of COVID-19 patient specimens and compared their sensitivity with that of RT-qPCR and infectious virus isolation. Although the RATs read the samples containing large amounts of virus as positive, even the most sensitive RAT read the samples containing small amounts of virus as negative. Moreover, all RATs tested failed to detect viral antigens in several specimens from which the virus was isolated. The current RATs will likely miss some COVID-19 patients who are shedding infectious SARS-CoV-2.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which emerged as a novel human pathogen in China at the end of 2019 [1] , is responsible for coronavirus disease 2019 (COVID-19), which causes symptoms such as cough and fever, severe pneumonia, and death. The WHO reported that more than 29 million cases of COVID-19, including approximately 900,000 deaths, have occurred as of 16 September 2020 (https://covid19.who.int/). To control the spread of SARS-CoV-2 infections, rapid identification and isolation of patients are required.

The gold standard for COVID-19 diagnosis is reverse transcription-quantitative PCR (RT-qPCR) using nasopharyngeal (N) swabs, throat (T) swabs, or saliva [2] . RT-qPCR kits that do not require viral RNA extraction and high-throughput RT-qPCR systems have been developed. Although such tests are widely utilized in public health laboratories and large well-equipped hospitals, they are unavailable in local clinics where patients who suspect they have COVID-19 often go first. Therefore, specimens need to be transported to and examined at sites that have RT-qPCR capability, which delays the test result and increases the anxiety of the suspected COVID-19 patients. To improve this situation, rapid antigen tests (RATs) for COVID-19, which does not require specific and expensive machinery, have been approved for clinical use in Japan and other countries, and the sensitivity of these tests has been compared with that of several kinds of RT-qPCR [3] [4] [5] [6] [7] [8] [9] . Although these RATs might be useful for the identification of COVID-19 patients in local clinics, their sensitivity is important in determining usage strategies. Here, we examined the sensitivity of four RATs available in Japan in August 2020 for the detection of isolated viruses. We also evaluated their effectiveness with several kinds of clinical specimens collected from COVID-19 patients and compared it with that of RT-qPCR and virus isolation.

Human samples were collected by following protocols approved by the Research Ethics Review Committee of the Institute of Medical Science, the University of Tokyo (approval number 2019-71-0201; 1 February 2020). Signed informed consent was obtained from all participants.

All experiments with SARS-CoV-2 were performed prior to 8 September 2020 in biosafety level 3 (BSL3) laboratories at the University of Tokyo, which were approved for such use by the Ministry of Agriculture, Forestry, and Fisheries, Japan.

Vero cells expressing human serine protease TMPRSS2 (Vero-TMPRSS2) [10] were maintained in DMEM containing 10% fetal calf serum (FCS), 1 mg/mL G418, 100 units/mL penicillin, 100 µg/mL streptomycin, and 5 µg/mL plasmocin prophylactic (InvivoGen, San Diego, CA, USA) and incubated at 37 • C under 5% CO 2 . We used these cells after clearance of mycoplasma.

SARS-CoV-2 (UT-NCGM02/Human/2020/Tokyo (NC02) and UT-HP072/Human/2020/Tokyo (HP72)] were isolated from clinical samples collected in February and August, respectively, and titrated in Vero-TMPRSS2 cells by performing plaque assays [11] .

Gargle lavage (n = 7), saliva (n = 27), throat (T) swab (n = 2), nasal vestibule swab (n = 1), nasopharyngeal (N) swab (n = 18), sputum (n = 4), and tracheal aspirate (n = 17) samples (Supplementary Materials Table S1 ) were collected from COVID-19 patients at several timepoints after onset. Swabs were soaked in BD universal viral transport medium; saliva, sputum, and tracheal aspirate samples were diluted in BD universal viral transport medium if needed. These media were used as test specimens. Gargle lavages were examined directly.

Viral RNA was isolated from the specimens by using the QIAamp Viral RNA Mini Kit (QIAGEN, Tokyo, Japan). One step RT-qPCR was performed using the LightCycler 96 System (Roche Diagnostics, Tokyo, Japan) according to the protocol of the National Institute of Infectious Disease, Japan [12] . A Cq value of >40 was considered a negative result.

The RATs listed in Table 1 were evaluated according to the procedures described in the manufacturers' instructions, using 75-7500 plaque formation unit (PFU) of stock virus in 50 µL of culture supernatant or 50 µL of specimen. Two independent experiments were performed with each sample. Tauns Laboratories Platinum-gold colloid 28.6 15 Japan a The lysed sample is dropped into the well and the reaction occurs inside a covered plastic body. b For all rapid antigen tests (RATs) tested, the test samples (50 µL) were mixed with lysis buffer (A). A part of the lysed sample (B) was then assayed. Input ratios were calculated using the following formula: volume B/(50 µL + volume A) × 100. c The time required to obtain the results is based on the manufacturer's instructions provided with the kit.

The specimens were inoculated into Vero-TMPRSS2 cells in 24 well-plates and the cells were incubated at 37 • C for 1 h. After removal of the inoculum, the cells were incubated in DMEM containing 5% FCS, 10 mM HEPES, 100 µg/mL gentamicin sulfate, and 2.5 µg/mL amphotericin B for 6 days at 37 • C. The appearance of cytopathic effects was checked for 6 days at least once a day.

We evaluated four RATs that were available in Japan in August 2020 (Table 1) . These RATs are immunochromatographic tests; therefore, their sensitivity is dependent on the binding kinetics of the monoclonal antibodies used in each RAT. The composition of the lysis buffer, the proportion of specimen in the analyte, and the method used to visualize the result also affect the sensitivity. As a result that the manufacturers do not disclose the composition of the lysis buffer, we could not compare this parameter. The amount of lysis buffer and specimen/lysis buffer mixture used for testing differed among the RATs tested. Accordingly, the percentages of the specimens used also differed: for Standard Q COVID-19 Ag, Espline SARS-CoV-2, QuickNavi -COVID19 Ag, and ImmunoAce SARS-CoV-2, the percentages of specimens used were 14.3%, 10.0%, 12.5%, and 28.6%, respectively ( Table 1 ). The method for visualization of the results also differed among these tests in that Standard Q COVID-19 Ag, Espline SARS-CoV-2, QuickNavi -COVID19 Ag, and ImmunoAce SARS-CoV-2 use color particles, alkaline phosphatase and its substrate, color latex, and platinum-gold colloids, respectively, to visualize the antigen-antibody immune-complexes. The visualized results are assessed by the human eye at 15-30, 30, 15, and 15 min, respectively, after adding the analyte. Although the types of specimens recommended for all four RATs are N swabs, we tested other types of specimens (see the footnote to Tables 2 and 3). 

To compare the sensitivity of these four RATs, two SARS-CoV-2 isolate stocks (NC02 and HP72) were diluted to the indicated PFU and then examined by RT-qPCR to determine the Cq value of each sample. (Table S2) . Standard Q COVID-19 Ag and ImmunoAce SARS-CoV-2 detected as little as 250 PFU of NC02, whereas Espline SARS-CoV-2 and QuickNavi -COVID19 Ag detected 500 PFU and 750 PFU of NC02, respectively (Table S2) . Standard Q COVID-19 Ag and Espline SARS-CoV-2 detected as little as 250 PFU of HP72, whereas QuickNavi COVID19 Ag and ImmunoAce SARS-CoV-2 detected 5000 PFU of HP72. These results show that the sensitivity for virus detection of Standard Q COVID-19 Ag and Espline SARS-CoV-2 was better than that of QuickNavi COVID19 Ag, and that virus detection by ImmunoAce SARS-CoV-2 differed depending on the isolate.

To evaluate the sensitivity of the four RATs for clinical specimens, 7 gargle lavages, 27 saliva, 2 throat (T) swabs, 1 nasal vestibule swabs, 18 nasopharyngeal (N) swabs, 4 sputum, and 17 tracheal aspirates were collected and their viral genomic RNA levels were examined by RT-qPCR. These 76 clinical specimens, which were divided into 6 groups based on their Cq value, were examined in the four RATs and subjected to virus isolation using Vero-TMPRSS2 cells ( Table 2 and Table S1 ). Standard Q COVID-19 Ag and Espline SARS-CoV-2 detected viral antigens in all specimens with Cq values lower than 22.5, whereas QuickNavi COVID19 Ag and ImmunoAce SARS-CoV-2 failed to detect viral antigens in several of these specimens. For specimens with a Cq value between 22.5 and 25.0, the sensitivity was on the order of Espline SARS-CoV-2, Standard Q COVID-19 Ag, ImmunoAce SARS-CoV-2, and QuickNavi COVID19 Ag ( Table 2 and Table S1 ). For the specimens with a Cq value greater than 25.0, the sensitivity of these four RATs was similar. These results indicate that the sensitivity of the four RATs is lower than that of RT-qPCR but similar to that of virus isolation. Overall, the sensitivity of Espline SARS-CoV-2 and Standard Q COVID-19 Ag was slightly better than that of ImmunoAce SARS-CoV-2; QuickNavi -COVID19 Ag had the lowest sensitivity.

We next plotted the results from the RATs and virus isolation, in accordance with the number of days after onset that the specimens were obtained, and the Cq values of the RT-qPCR (Figure 1 ). Although all of the RATs efficiently detected viral antigens in the specimens with low Cq values, the results from the RATs could be different for two specimens with similar Cq values (i.e., some were RAT-positive, whereas others were RAT-negative). Several specimens with Cq values higher than those of specimens that were RAT-negative, but with the above-described low Cq values (as indicated by cyan arrows in Figure 1 ) were found to be RAT-positive. These results suggest that the viral RNA and protein content may have differed between the specimens or that some samples may have contained inhibitors or enhancers for the antigen-antibody reaction of the RATs. Importantly, several specimens from which the virus was isolated were negative with all RATs tested, indicating that these RATs could potentially fail to diagnose patients shedding infectious SARS-CoV-2. Since the type of specimen likely affects the sensitivity of RATs, we assessed our results according to specimen type (gargle lavage, saliva, T swab, nasal vestibule swab, N swab, sputum, All specimens were also tested by RT-qPCR and subjected to virus isolation. Open red circles indicate specimens from which virus was isolated; open black circles indicate specimens from which virus was not isolated. Circles filled in black indicate both results were positive, those filled in gray indicate that one was positive, no color indicates both were negative.

Since the type of specimen likely affects the sensitivity of RATs, we assessed our results according to specimen type (gargle lavage, saliva, T swab, nasal vestibule swab, N swab, sputum, and tracheal aspirate). Although the Cq value varied between specimens from 18.8 to 36.0, gargle lavage, T swabs, and nasal vestibule swabs showed higher Cq values than the other types of specimens (Table 3) . All four RATs detected virus antigens in one sputum sample, although no virus was isolated from 4 sputum samples tested. All four RATs failed to detect virus antigens in all 7 gargle lavages, 2 T swabs, and 1 nasal vestibule swabs, although the virus was isolated from one T swab sample. Since N swabs and saliva are suitable specimens for diagnosis in clinics, we plotted the results from the RATs and virus isolation based on the number of days after onset that the specimens were collected and the Cq values of the RT-qPCR ( Figure 2 ). Standard Q COVID-19 Ag, Espline SARS-CoV-2, and ImmunoAce SARS-CoV-2 efficiently detected virus antigens in N swabs and saliva with low Cq values (Cq < 25) that were collected at early timepoints after onset. 

Here we evaluated the sensitivity of four RATs available in Japan in August 2020. The overall sensitivity of Standard Q COVID-19 Ag and Espline SARS-CoV-2 was better than that of ImmunoAce 

Here we evaluated the sensitivity of four RATs available in Japan in August 2020. The overall sensitivity of Standard Q COVID-19 Ag and Espline SARS-CoV-2 was better than that of ImmunoAce SARS-CoV-2 and QuickNavi COVID19 Ag. For specimens such as saliva and swabs, Standard Q COVID-19 Ag, Espline SARS-CoV-2, and ImmunoAce SARS-CoV-2 had similar detection sensitivities. These three kits detected viral antigens in more than half of the specimens whose Cq values of RT-qPCR were less than approximately 25. Therefore, these RATs may be suitable for the detection of COVID-19 in individuals who are shedding a large amount of SARS-CoV-2; that is, they may be useful to identify patients with a high likelihood of transmitting the virus to others.

All four RATs failed to detect viral antigens in several specimens from which virus was isolated. This finding indicates that the current RATs are likely to miss some COVID-19 patients who are shedding infectious SARS-CoV-2. However, it is unclear to what extent such patients would transmit virus to others. Further studies are required to address this point. In addition, the present study used swab specimens that were soaked in transport medium; if the specimens were directly soaked in lysis buffer, the sensitivity of the RATs might be improved. The recommended samples for these RATS are nasopharyngeal swabs not saliva, and some of the saliva samples from which SARS-CoV-2 was isolated gave negative results with all of the RATs tested here. Nonetheless, these RATs need to be improved with respect to their sensitivity.

In this study, we focused on the sensitivity of four RATs; we did not evaluate the false-positive rates of these kits. Therefore, studies on false-positive rates should be performed in the future to determine the specificity of these and other RATs.

Supplementary Materials: The following are available online at http://www.mdpi.com/1999-4915/12/12/1420/s1, Table S1 : List of specimens and results, Table S2 : Sensitivity of rapid antigen tests for stock viruses. 

A Novel Coronavirus from Patients with Pneumonia in China

Interpreting Diagnostic Tests for SARS-CoV-2

Clinical Evaluation of Self-Collected Saliva by Quantitative Reverse Transcription-PCR (RT-qPCR), Direct RT-qPCR, Reverse Transcription-Loop-Mediated Isothermal Amplification, and a Rapid Antigen Test to Diagnose COVID-19

Evaluation of rapid antigen test for detection of SARS-CoV-2 virus

Evaluation of a Rapid Diagnostic Assay for Detection of SARS-CoV-2 Antigen in Nasopharyngeal Swabs

Evaluation of a novel antigen-based rapid detection test for the diagnosis of SARS-CoV-2 in respiratory samples

Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis

Development and Potential Usefulness of the COVID-19 Ag Respi-Strip Diagnostic Assay in a Pandemic Context

Implementation of rapid SARS-CoV-2 antigenic testing in a laboratory without access to molecular methods: Experiences of a general hospital

Enhanced isolation of SARS-CoV-2 by TMPRSS2-expressing cells

Syrian hamsters as a small animal model for SARS-CoV-2 infection and countermeasure development

Development of Genetic Diagnostic Methods for Detection for Novel Coronavirus 2019(nCoV-2019) in Japan

We thank Noriko Kinoshita, Shigehiro Hagiwara, Daisuke Arai, Hidenori Takahashi, Genta Nagao, and Shunichiro Konishi, Hiroko Shibata, Osamu Takahashi, Takashi Momoda, Hiromi Isshiki for assistance of experiments and Susan Watson for editing the manuscript.

Yoshihiro Kawaoka obtained funds to organize a symposium, "Influenza and Other Infections" in 2019 from TAUNS Laboratories, Inc. Kei Yamamoto has received grant support from Fujirebio, Inc.