key: cord-0839356-b8ufz9ja authors: Nalumansi, Aminah; Lutalo, Tom; Kayiwa, John; Watera, Christine; Balinandi, Stephen; Kiconco, Jocelyn; Nakaseegu, Joweria; Olara, Denis; Odwilo, Emmanuel; Serwanga, Jennifer; Kikaire, Bernard; Ssemwanga, Deogratius; Nabadda, Susan; Ssewanyana, Isaac; Atwine, Diane; Mwebesa, Henry; Bosa, Henry Kyobe; Nsereko, Christopher; Cotten, Matthew; Downing, Robert; Lutwama, Julius; Kaleebu, Pontiano title: Field Evaluation of the Performance of a SARS-CoV-2 Antigen Rapid Diagnostic Test in Uganda using Nasopharyngeal Samples date: 2020-10-30 journal: Int J Infect Dis DOI: 10.1016/j.ijid.2020.10.073 sha: 15325dbdc102b6bac0b65c31c6b2a16a795c71ef doc_id: 839356 cord_uid: b8ufz9ja OBJECTIVES: There is a high demand for SARS-CoV-2 testing to identify COVID-19 cases. Real-time, quantitative PCR (qRT-PCR) is the recommended diagnostic test but a number of constraints, including cost prevent its widespread implementation. The aim of this study was to evaluate a low cost, easy-to-use rapid antigen test for diagnosing COVID-19 at the point-of-care. METHODS: Nasopharyngeal swabs from suspect COVID-19 cases and from low-risk volunteers were tested on the STANDARD Q COVID-19 Ag Test and results compared with the qRT-PCR results. RESULTS: 262 samples were collected including 90 qRT-PCR positives. The majority were from males (89%) with a mean age of 34 years and only 13 (14%) of the positives were mildly symptomatic. Sensitivity and specificity of the antigen test were 70.0% (95% CI: 60 - 79) and 92% (95% CI: 87 - 96) respectively; diagnostic accuracy was 84% (95% CI: 79 - 88). The antigen test was more likely to be positive in samples with qRT-PCR Ct values ≤29 reaching a sensitivity of 92%. CONCLUSIONS: The STANDARD Q COVID-19 Ag Test performed less than optimally in this evaluation. However, the test may still have an important role to play early in infection when timely access to molecular testing is not available but results should be confirmed by qRT-PCR. Public health experts recommend that in the absence of treatment for coronavirus disease 2019 (COVID-19) or a vaccine for severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2), speedy and accurate testing, followed by case identification, isolation and contact tracing are the best approaches to contain this new disease. COVID-19 was declared a pandemic by the World Health Organization (WHO) on March 11 th , 2020. Identifying those infected with the virus is complicated by the high overlap between COVID-19 clinical symptoms and other respiratory infections and by the fact that many infected individuals are asymptomatic (Yang et al., 2020; Lavezzo et al., 2020) . Accurate diagnostic testing for case identification, quarantine and contact tracing is therefore key in managing this pandemic. The global demand for testing has put a substantial strain on governments and institutions. The gold-standard diagnostic test recommended by WHO (WHO 2020a) is the real-time, quantitative reverse-transcription polymerase chain-reaction (qRT-PCR), a nucleic acid amplification test (NAAT), aimed at the detection of viral RNA. In Uganda, SARS-CoV-2 qRT-PCR testing was started at the Uganda Virus Research Institute (UVRI) in early February 2020 using the Berlin protocol . All testing for Uganda was carried out at the UVRI for the initial two months before decentralization. The first confirmed case was detected on 21 st March 2020. By mid-July, Uganda had conducted 230,680 tests, the highest number in East Africa and had reported 1040 COVID-19 infections. The majority of these infections were imported by truck drivers at the border crossings with neighbouring countries. These point-of-entry (POE) cases need quick result-turnaround times for early interventions to avoid the transmission of infection to local communities and to limit the disruption to commercial transportation. qRT-PCR testing is usually performed in designated, specialized laboratories requiring welltrained staff and the test typically takes 4-6 hours to complete. The time taken to ship clinical samples to the laboratory and return results to health facilities has an overall turnaround time of 24-48 hours. This leads to prolonged periods of isolation for suspect cases and delayed contact tracing which could further increase the spread of the infection within the country. The cost for qRT-PCR is also prohibitive and is further complicated by the global procurement challenges, most reagents currently taking 1-2 months for delivery. To overcome these constraints, simple, low-cost and easy-to-use rapid antigen diagnostic tests are urgently required at the point-of-care (POC). There are now a number of antibody and antigen rapid diagnostic tests (RDTs) available on the market that can be used by staff with minimal training and these are attractive options for the decentralization of testing. Antibody tests are used for surveillance and epidemiological research and to identify recent or past infection while antigen RDTs could be an alternative to qRT-PCR in detecting acute infection. Moreover, WHO in the recent guidance has proposed settings where antigen RDTs can be used in patient management (WHO 2020b). However, there have been few field evaluations of these antigen RDTs ( In Uganda, all new molecular testing kits and immunoassays introduced to the Uganda market must undergo an in-country laboratory verification at UVRI, a designated WHO and Africa CDC SARSCoV-2 reference laboratory, before being recommended to the Ministry of Health (MOH) for use in the country. WHO also advises that before tests are recommended, they should be verified in appropriate populations and settings (WHO Bulletin., 2017; ECDC., 2002). To date, there is no immuno-assay which has been recommended for use in Uganda. Here we report on the evaluation of the performance characteristics of the STANDARD Q COVID-19 Ag Test (SD Biosensor, Gyeonggi-do, 16690, Korea) compared with the qRT-PCR (Berlin protocol) using nasopharyngeal swabs. Our experience may prove useful to other reference laboratories in an effort to meet the SARS-CoV-2 diagnostic demands. This was a cross sectional, prospective, un-blinded verification of the performance of the STANDARD Q COVID-19 Ag Test. Participants were recruited at the regional referral hospitals (RRHs) of Arua, Entebbe, Fort Portal, Gulu, Jinja, Lira, Masaka, Mbale and at Mulago National Referral Hospital. The non-case controlswere volunteers at the Kasenyi Military Barracks and at the UVRI Clinic (Appendix 1). Sample collection was coordinated by a team of laboratory staff from UVRI. Two nasal swabs were collected by laboratory personnel at the COVID-19 treatment facilities while taking all necessary biosafety precautions -a swab was obtained from each nostril. One swab was tested immediately at the facility using the STANDARD Q COVID-19 Ag Test and the result interpreted according to the manufacturer's guidelines and recorded on an Excel worksheet. The second swab was preserved in specimen transport media and transported at 4℃ to the UVRI laboratory for extraction and qRT-PCR testing (Corman et al., 2020). All testing followed procedures described in the UNCST-approved protocol "Uganda Virus Research Institute; Performance Evaluation for CVID-19 Diagnostic Tests" (http://www.uvri.go.ug/projects/covid-19). The STANDARD Q COVID-19 Ag Test is a rapid chromatographic immunoassay for the qualitative detection of SARS-CoV-2 specific antigens present in the human nasopharynx. According to the J o u r n a l P r e -p r o o f manufacturer's 'Information for Use' (IFU), results are available within 30 minutes. All necessary reagents are provided by the manufacturer to perform the assay and no assay-specific, specialized equipment is needed. According to the IFU, the assay kits are stable when stored at 2-30℃. RNA extraction: RNA was extracted from clinical samples using the viral RNA mini kit (QIAGEN, Hilden, Germany). qRT-PCR (Berlin Protocol): Oligonucleotides were synthesised and provided by Metabion (http://www.metabion.com). Thermal cycling was performed at 55 °C for 10 minutes for reverse transcription, followed by 95 °C for 3 minutes and then 45 cycles at 95 °C for 15 seconds and at 58 °C for 30 seconds. qRT-PCR was performed on an Applied Biosystems platform. Sensitivity was calculated as the number of specimens identified as positive by the STANDARD Q COVID-19 Ag test, divided by the number of specimens identified as positive by the qRT-PCR reference assay, expressed as a percentage. Specificity was calculated as the number of specimens identified as negative by the STANDARD Q COVID-19 Ag test, divided by the number of specimens identified as negative by the qRT-PCR reference assay, expressed as a percentage. Accuracy: This was calculated as the proportion of STANDARD Q COVID-19 Ag Test results that were in agreement with the qRT-PCR results (positive and negative), expressed as a percentage. The sensitivity, specificity and accuracy calculations were performed using the proportion command in STATA ® 15 which also generated the 95% Confidence Intervals. We determined the relationship between viral load, as measured by the qRT-PCR Ct value and antigen detection. Ct values were categorized as strongly positive (Ct ≤ 29) (indicative of abundant target nucleic acid in the sample), moderately positive (Ct 30-37) and weakly positive (Ct 38-39) and compared with the STANDARD Q COVID-19 Ag Test results. The evaluation protocol was reviewed and approved by UVRI's Research Ethics Committee (REC) and the Uganda National Council for Science and Technology (UNCST). Specimens were unlinked to personal identifiers and results could not be traced to individual patients. Consent to participate and to store samples for future use was also sought. A total of 90 COVID-19 cases and 172 controls (total 262) were included in this evaluation. The majority were males (89%) and the overall mean age was 34 years (95% CI: 32 -35; Table 1 ) Test result distribution: The distribution of STANDARD Q COVID-19 Ag Test results against the qRT-PCR results are presented in Table 2 . Overall, there were 76 (29.0%) specimens that were antigen-positive and 186 (71%) specimens that were antigen-negative. The STANDARD Q COVID-19 Ag Test showed a sensitivity of 70% (95% CI: 60 -79) and a specificity of 92% (95% CI: 87 -96). The STANDARD Q COVID-19 Ag Test showed an accuracy of 84% (95% CI: 79 -88). The false positive rate was 8% (95% CI: 4 -13) and the false negative rate was 30% (95% CI: 21 -40). There were no factors associated with false positives, these cut across age categories and RRHs. The association between the Ct values and STANDARD Q COVID-19 Ag test result is shown in Table 3 . The STANDARD Q COVID-19 Ag Test, not unexpectantly, was more likely to be positive when a specimen had abundant target nucleic acid -92% of specimens with a strong positive qRT-PCR result were positive on the STANDARD Q COVID-19 Ag test. Only 50% of specimens categorized as moderate and low positive samples by qRT-PCR were positive by the antigen test specimens (p<0.01). WHO has recently issued interim guidance on the use of antigen RDTs for patient management. At a minimum, the Ag-RDTs should correctly identify significantly more cases than they miss (sensitivity ≥80%) and have very high specificity ( ≥97-100%) (WHO 2020b). If any of the antigen detection tests that are under development demonstrate adequate performance, they could potentially be used as triage tests to rapidly identify patients who are very likely to have COVID-19 thereby reducing or eliminating the need for expensive NAAT (ICMR 2020). In this evaluation of the STANDARD Q COVID-19 Ag Test, a commercial antigen RDT, we found that the test had a sensitivity and specificity of 70% and 92% respectively. Overall, combining both the positive and negative samples, the test had an accuracy of 84%. Sensitivity in this evaluation was lower than that reported by the manufacturer (70% compared with 84.38%) and the difference was statistically significant (p<0.001). Similarly, specificity in this evaluation was significantly lower than that reported by the manufacturer (92% compared with 100%). These discrepancies may be due in part to the limitations of this study discussed below. With a sensitivity of 70%, the STANDARD Q COVID-19 Ag Test would not detect 30 out of 100 qRT-PCR positive samples. Likewise, with a specificity of 92%, the STANDARD Q COVID-19 Ag Test would find that 8 out of 100 qRT-PCR negative samples were positive. The need for timely testing is crucial in order to contain the pandemic. Currently most qRT-PCRbased testing is conducted in designated, specialized laboratories which are far from many sample collection sites or patients and their contacts, leading to long turnaround times for results reporting, delayed isolation and contact tracing and thereby increasing the risk of further spread of the virus within the country. Decentralization of testing using mobile PCR laboratories and POC GeneXpert platforms has been introduced in Uganda to shorten the result turnaround time. However, the challenges of reagent supply, cost and low testing capacity at times still remain. The introduction at the POE of an antigen RDT with good performance would be a significant improvement on current practice. The results obtained in this study show that the SB Biosensor antigen RDT has a less than optimal performance -the ideal test would have a sensitivity >95% and a specificity of 100%. The STANDARD Q COVID-19 Ag Test (Gyeonggi-do, 16690, Korea) had better performance (sensitivity; 84.38%, specificity; 100%) during validation by the manufacturer in Malaysia (STANDARD Q COVID-19 Ag Test IFU) and in a two-site evaluation in India (ICMR 2020) (sensitivity; 50.6% and 84.0%, specificity; 99.3% and 100%). Ag Respi-Strip (Coris Bioconcept, Gembloux, Belgium) was found to have good specificity (99.5% -100%) but poor sensitivity (30.2% -57.6%) compared to qRT-PCR. Another RDT, the BIOCREDIT COVID-19 Ag test (RapiGEN Inc., Gyeonggi-do, 14119, Korea) (Mak et al., 2020) also showed low sensitivity (11.1% -45.7% with specimens from the nasopharynx, throat, saliva and sputum) the authors concluding that this test can only be used as adjunct to qRT-PCR test because of the potential for false-negative results. On the other hand, there are two reports showing good performance of another antigen test, the Fluorescence Immunochromatographic SARS-CoV-2 Antigen Test (Bioeasy Biotechnology Co., Shenzhen, China). In one study of 127 participants in Chile (Porte et al., 2020), the overall sensitivity and specificity were 93.9% and 100% respectively, with an accuracy of 96.1%. In the second study of 239 participants in China (Diao et al., 2020) , the overall sensitivity and specificity were 67.8% and 100% respectively. Similar to our findings, sensitivity in both studies was significantly higher in samples with high viral loads. Even with a less than optimal performance, the STANDARD Q COVID-19 Ag test could still play a role in triage, but with subsequent selective testing by qRT-PCR, in situations where quick isolation of cases is required, for example with symptomatic cases or with 'high-risk' truck drivers at border crossings. In such a situation, all individuals identified as COVID-19-positive would require individualized isolation until the qRT-PCR results were available, to avoid contact with true positives by the few antigen false-positives who are in fact not infected. This approach can reduce the time in isolation for those infected and simplify the logistics for contact tracing; however, this comes with the additional cost of the antigen test. India has taken a different approach (ICMR 2020). Following their evaluation of the STANDARD Q COVID-19 Ag Test, the Indian Council of Medical Research recommended that for high-risk groups in containment zones or hotspots and healthcare settings, 'suspected individuals who test negative for COVID-19 by rapid antigen test should definitely be tested sequentially by RT-PCR to rule out infection, whereas a positive test should be considered as a true positive and does not need reconfirmation by RT-PCR'. There were limitations to this prospective and un-blinded study. Those evaluating the antigen RDT knew which participants were likely to be infected as opposed to uninfected in most cases. This may have introduced bias into their interpretation of the antigen test result, especially where RDT bands were weak. The staff taking swab samples were different at each health facility and had minimal training which may have introduced variability in the content or yield of swab material. Moreover, two swabs were taken from each subject, one from each nostril. One was tested immediately on the antigen RDT and the other placed in transport media for later qRT-PCR testing at UVRI. Effectively the two swabs can be regarded as two different samples. The date of exposure was not known for any of the participants and only 13 of the cases had date of symptom onset since the majority of participants were asymptomatic (86%). Day zero was taken as the date the first swab was taken. The analysis would have benefitted from knowing the date of exposure since viral load increases over the first 1 -2 weeks (Wolfel et al., 2020; Zou et al., 2020) and appears to correlate with the antigen test performance. What was observed was the better sensitivity for samples with low Ct values presumed to have higher viral loads. Due to logistical constraints, some swab specimens were transported to UVRI in normal saline rather than viral transport media (VTM). Although WHO recommends the use of normal saline in the absence of the VTM, the effect on qRT-PCR sensitivity is not well documented. The qRT-PCR result was taken as the gold standard. However, both the qRT-PCR and antigen RDT will miss some infections particularly those mentioned above as limitations in the collection and transport of swabs. A more significant source of error in this evaluation perhaps lies in deciding the qRT-PCR Ct cut-off value for a positive sample. Results for specimens with a Ct value near the cut off should perhaps be regarded as provisional and ideally should be repeated on a fresh sample until such time as the Ct cut-off value for a positive result can be defined more precisely. Going forward it will be important to develop standard operating procedures that address the limitations mentioned above in the pre-analytical, analytical and post-analytical phases of testing. The STANDARD Q COVID-19 Ag Test had a less than optimal performance in this evaluation. However, the test may still have an important role to play early in infection when timely access to molecular testing is not available but results should be confirmed by qRT-PCR. Suppression of a SARS-CoV-2 outbreak in the Italian municipality of Vo' 3 WHO 2020a Laboratory testing strategy recommendations for COVID-19. Interim guidance Detection of 2019 novel coronavirus (2019-nCoV) by realtime RT-PCR WHO 2020b Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays Interim guidance 11 September 2020 COVID-19: Laboratory and diagnosis Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis Evaluation of novel antigen-based rapid detection test for the diagnosis of SARS-CoV-2 in respiratory samples Evaluation of a rapid diagnostic assay for detection of SARS CoV-2 antigen in nasopharyngeal swab. JCM Accepted Manuscript Posted Online 13 Ag Respi-Strip Diagnostic Assay in a Pandemic Context. 2020 Front Diagnosis of acute respiratory syndrome coronavirus 2 infection by detection of nucleocapsid protein. medRxiv Evaluation of rapid antigen test for detection of SARS-CoV-2 virus ICMR Advisory on Use of Rapid Antigen Detection Test for COVID-19 14 ECDC Diagnostic testing and screening for SARS-CoV SD Biosensor STANDARD Q COVID-19 Ag Test IFU: Available from Virologic assessment of hospitalized patients with COVID-2019 SARS-CoV-2 viral load in upper respiratory specimens of infected patients We thank the individuals who provided the specimens including those at the different COVID-19 isolation centres. We thank the staff at the regional referral hospitals of Arua, Entebbe, Fort Portal, Gulu, Jinja, Lira, Masaka, and Mbale as well as staff at Mulago National Referral Hospital, the Kasenyi Military Barracks, CDC Uganda and the UVRI Clinic. We also thank members of the COVID-19 Ministry of Health Scientific Committee and staff of the Expanded Programme on Immunization at UVRI who collected additional samples from symptomatic patients.The Korea Foundation for International Healthcare (KOFIH) provided the SD Biosensor STANDARD Q COVID-19 Ag kits. We acknowledge the Uganda Government funding to UVRI. The procurement of the SARS-CoV2 qRT-PCR kits was funded by the WHO, Africa CDC, Jack Ma TL, CW, BK, DS, DA, HM, PK conceived and designed the study and/or wrote and proof-read the manuscript. AM, DO, EO, JS, SN, IS, HK, MC and CN provided specimens and/or demographic data and/or conducted antigen tests and/or interpreted the data. JKa, SB, JKi, JN, CN and JL conducted the molecular testing. TL handled the data and analysis. TL, RD and PK revised/edited the manuscript for intellectual content.All authors reviewed the manuscript and gave final approval for submission No specific funding was received from funding bodies in the commercial, public or not-for-profit sectors. All authors declare no competing interests.