key: cord-337799-mc1oqhf4 authors: Mak, Gannon CK; Lau, Stephen SY; Wong, Kitty KY; Chow, Nancy LS; Lau, CS; Lam, Edman TK; Chan, Rickjason CW; Tsang, Dominic NC title: Analytical sensitivity and clinical sensitivity of the three rapid antigen detection kits for detection of SARS-CoV-2 virus date: 2020-10-29 journal: J Clin Virol DOI: 10.1016/j.jcv.2020.104684 sha: doc_id: 337799 cord_uid: mc1oqhf4 BACKGROUND: Numerous rapid antigen detection (RAD) kits for diagnosing COVID-19 patients are available in the market recently. OBJECTIVE: To compare analytical sensitivity and clinical sensitivity for the three commercially available RAD kits. STUDY DESIGN: Analytical sensitivity for the detection of SARS-CoV-2 virus was determined by limit of detection (LOD) using PCR as a reference method. Clinical sensitivity was evaluated by using respiratory specimens collected from confirmed COVID-19 patients. RESULTS: The LOD results showed that the three RAD kits varied 102 to 105 fold less sensitive than RT-PCR. Clinical sensitivity of RAD kits ranged from 22.9% to 71.4% for detecting specimens from COVID-19 patients. CONCLUSIONS: Although RAD kits were less sensitive than RT-PCR, understanding the clinical characteristics of different RAD kits can guide us to obtain suitable specimens for testing. The likelihood of positive results for RAD kits will be higher. The rapid diagnosis of COVID-19 patients is essential to reduce the disease spread. Besides RT-PCR, rapid antigen detection (RAD) kits for qualitative determination of SARS-CoV-2 antigen are available. RAD kits are less expensive than RT-PCR. In addition, the test results of RAD kits can be available without specialized instrument and interpreted within 30 min. Currently, WHO does not recommend specific commercial RAD kit for patient care. However, researches of different RAD kits are encouraged especially for the performance of sensitivity [1, 2] . We previously evaluated a RAD kit for SARS-CoV-2 on May 2020 [3] . Since then, many new RAD kits are commercially available. Data on the performance of RAD kits varied between different studies and different brands [4] [5] [6] [7] [8] [9] . The purpose of this evaluation is to assess analytical sensitivity of the three SARS-CoV-2 RAD kits by means of limit of detection (LOD) using a set of serial tenfold dilution samples; and The RAD kits selected in the present study were all commercially available in Hong Kong. These kits were either evaluated in previous studies or demonstrated good clinical performance mentioned in the kit inserts. The three kits evaluated were: (1) These three kits employ a lateral flow test format to detect viral antigen by the immobilized coated SARS-CoV-2 antibody on the device. The test results can be interpreted without a reader at 15 min. The intended use for these kits is for the detection of SARS-CoV-2 virus in either nasopharyngeal swabs or secretions. The procedures were carried out according to manufacturer's instructions. For the NADAL, since our specimens were placed in viral transport media or phosphatebuffered saline, 350μL specimen volume was mixed with 10 drops of extraction buffer (personal communication with Nal Von Minden GmbH). The subsequent procedures were carried out according to the manufacturer's instructions. To determine LOD between different kits, a respiratory specimen was selected to perform a serial tenfold dilution. The selected specimen was nasopharyngeal aspirate and throat swab (NPA & TS) which was obtained from the Hong Kong COVID-19 patient, hCoV-19/Hong Kong/VM20030475/2020. To minimize the number of freezethaw cycle, the three kits were tested in parallel for this set of serial dilution at the same time. The BIOCREDIT COVID-19 Ag (RapdGEN Inc. Korea) kit was also tested as a reference method simultaneously. Virus concentrations in each dilution were estimated from cycle threshold (Ct) value as described [3] . J o u r n a l P r e -p r o o f Ideally, the specimens selected should be tested in parallel for the three kits. In order to evaluate the three kits simultaneously, a total specimen volume of 800μL is needed (Coris=100μL, NADAL=350μL, Standard Q=350μL). Due to the limited quantity for each specimen, the same specimen cannot be tested for all kits. Since both Coris and Standard Q shared similar operating procedures, they were tested together using the same set of specimens. Another set of specimens were tested for NADAL. for these two sets of specimens were roughly the same, specimens of various Ct values up to ∼34 were selected (which is close to the limit of detection for RT-PCR). The technicians performing the RAD kits were blinded to the RT-PCR results. All of the kits evaluated were not involved using readers for determining a positive or negative result. Visual output of the cassettes was read by two technicians. In case of disagreement, the visual output was read by the third technician. J o u r n a l P r e -p r o o f The LOD of the reference RAD kit, BIOCREDIT COVID-19 Ag was 10 5 fold less sensitive than RT-PCR (BIOCREDIT COVID-19 Ag: 10− 2 ; RT-PCR: 10− 7 ) which was concordant to our previous study in May 2020. As shown in Table 1 , the three RAD kits shared the same LOD at 10− 2 , they were BIOCREDIT COVID-19 Ag, Coris and NADAL. The LOD of Standard Q was 10− 5 . The corresponding Ct values for LOD at 10− 2 and 10− 5 were 18.44 and 28.67 respectively. Although viral culture was not performed in the present study, the Standard Q was 10 2 fold less sensitive than RT-PCR, it corresponded to the LOD of viral culture based on our results reported previously. Since the cut-off of 18.57 for BIOCREDIT COVID-19 Ag was utilized in our previous study in May 2020, it was also used in the present study for consistency to classify specimens as 'high viral load'. Specimens with Ct values between 18.57 and 28.67 were classified as 'normal viral load' while specimens >28.67 were classified as 'low viral load'. Of the 280 specimens tested, 72, 132 and 76 specimens were classified as 'high viral load', 'normal viral load' and 'low viral load' specimens respectively. The ratio was approximately 1:2:1 which was similar to the viral load distribution of specimens collected by PHLSB. RAD kits showed high sensitivity (77.8% to 100%) for high viral load specimens but low sensitivity (0 to 11.1%) for low viral load specimens. For normal viral load specimens, one kit was capable of having sensitivity of ≥75% while the other two kits ranged from 6 In the cross-reactivity test using virus isolates, all were tested negative by the RAD test. The purpose of this study is to evaluate if the commercially available RAD kits are J o u r n a l P r e -p r o o f suitable for diagnosing SARS-CoV-2 infection, however, ranking of RAD kits is not our objective. Our study demonstrated a good correlation between LOD and clinical sensitivity. It means that the sensitivity of RAD kits is viral load dependent. Viral load depends on whether the patients are symptomatic and the course of the disease. It should be noted that although our data indicated that RAD kits were capable of detecting SARS-CoV-2 virus in throat saliva, majority of invalid results were found in throat saliva which should be due to the viscous nature of throat saliva. Similar findings were also observed in our previous study [3] . Throat saliva is not recommended as a specimen of choice where the collection of other types of specimens is available. Understanding the clinical characteristics of different RAD kits can guide us to obtain suitable specimens for testing. The performance of RAD kits could be tailored by targeting a specific patient group. Specimens obtained ≤5 days after symptom onset are most likely contain high viral loads [10] . We have analyzed the proportion of specimen for the two cut-offs, Ct ≤18.57 and Ct ≤28.67, according to days after symptom onset from ≤day 1 to ≤day 7. A total of 3932 specimens were available for analysis. These specimens comprised throat saliva, NPS & TS, NPA & TS, NPS collected from 31 Jan 2020 to 13 Sep 2020. Information of date of symptom onset for J o u r n a l P r e -p r o o f each specimen was collected [11] . We found that viral loads of some specimens collected early after symptom onset have similar viral loads for specimens collected late after symptom onset. It means that if we set a cut off by means of ≤day X after symptom onset for performing RAD kits, we will miss another group of specimens having a similar high viral load. We found that for Ct ≤18.57, testing specimens obtained ≤5 days after symptom onset will likely to match the high viral load specimens while keeping the minimum proportion of specimens having a similar high viral load that were missed. For Ct ≤28.67, specimens obtained ≤7 days after symptom onset are optimum (Supplementary Table s ). This observation was concordant to the WHO guideline that symptomatic cases should be tested within the first 5-7 days of illness [2] . The Standard Q might be suitable for use as an aid to early diagnosis of SARS-CoV-2 infection in terms of analytical and clinical sensitivity. This kit is listed in the 'WHO Emergency Use Listing for In vitro diagnostics (IVDs) Detecting SARS-CoV-2' [12] . The exact time period after symptom onset for use is not mentioned by the Standard Q. A recent study showed that the sensitivity can be decreased from 83.9% to 76.3% when specimens obtained ≤5 and ≤7 days after symptom onset respectively [13] . Our data showed that although the prevalence of specimens with Ct ≤28.67 decreased J o u r n a l P r e -p r o o f from 81.8% to 79.9% when specimens obtained ≤5 and ≤7 days after symptom onset respectively, the number of specimens with Ct ≤28.67 that were skipped for testing was decreased from 18.2% to 9.4% (Table S2) . Therefore, we recommended specimens obtained ≤7 days after symptom onset for use with the Standard Q. Then, the RAD kit can serve as a COVID-19 filter (filtered out of the infected persons and prevent spread to the others). The balance between turnaround time and sensitivity should be considered. A diagnostic test with high analytical sensitivity can still be failed to be a COVID-19 filter [14] . The limitation of this study is to use Ct value to estimate viral load. Ct values for a given target RNA can vary between RT-PCR assays and both intra-and inter-PCR runs. However, the same in-housed developed RT-PCR assay is consistently used for diagnosing SARS-CoV-2 infection in PHLSB and also our evaluations, variations of Ct values can be minimized. Although Ct is a semiquantitative value, it can also be used for guiding infection control and public health decisions [15, 16] . It is expected that many RAD kits for diagnosing SARS-CoV-2 infection are emerged in this exploding field. Access to laboratory services based on rapid diagnostic tests alone is common in developing countries [17] . With the lack of regulatory policies, J o u r n a l P r e -p r o o f laboratories took the responsibility for governance and quality control for rapid diagnostic tests [18] . The difficulty of evaluating RAD kits is limited by collecting a large number of positive specimens of different types. The LOD can predict the performance of RAD tests and it has been utilized in RAD kits for diagnosing influenza [19] . In future, the LOD approach will be performed first in our setting. When the LOD results demonstrate better results than the reference method, a full evaluation using clinical specimens of various concentrations will be performed. It should be noted than when determining clinical sensitivity, if only specimens of high viral load were selected, every kit will have a 100% clinical sensitivity [20] . In conclusion, adoption of RAD kit for clinical use should be guided by evidences. When the study is well-performed to understand the clinical characteristics, the RAD kit will be safe to use. None. J o u r n a l P r e -p r o o f Advice on the use of point-of-care immunodiagnostic tests for COVID-19. Scientific brief Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays. Interim guidance Evaluation of rapid antigen test for detection of SARS-CoV-2 virus Development and Potential Usefulness of the COVID-19 Ag Respi-Strip Diagnostic Assay in a Pandemic Context. Front Med (Lausanne) Evaluation of a Rapid Diagnostic Assay for Detection of SARS-CoV-2 Antigen in Nasopharyngeal Swabs Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis Implementation of rapid SARS-CoV-2 antigenic testing in a laboratory without access to molecular methods: Experiences of a general hospital Evaluation of Two Rapid Antigen Tests for Detection of SARS-CoV-2 Virus Urgent need of rapid tests for SARS CoV-2 antigen detection: evaluation of the SD-Biosensor antigen test for SARS-CoV-2 Spatial and temporal dynamics of SARS-CoV-2 in COVID-19 patients: A systematic review and meta-analysis Latest situation of cases of COVID-19 WHO. WHO Emergency Use Listing for In vitro diagnostics (IVDs) Detecting SARS-CoV-2 Clinical evaluation of BD Veritor SARS-CoV-2 point-of-care test performance compared to PCR-based testing and versus the Sofia 2 SARS Antigen point-of-care test Rethinking Covid-19 Test Sensitivity -A Strategy for Containment Predicting Infectious Severe Acute Respiratory Syndrome Coronavirus 2 From Diagnostic Samples A call for diagnostic tests to report viral load Point-of-care tests for diagnosing infections in the developing world How are rapid diagnostic tests for infectious diseases used in clinical practice: a global survey by the International Society of Antimicrobial Chemotherapy (ISAC) Epub ahead of print Comparative analytical sensitivities of six rapid influenza A antigen detection test kits for detection of influenza A subtypes H1N1, H3N2 and H5N1 Response to letter of concern by Oladimeji and Pickford of PrimerDesign