key: cord-0431384-pujspk2h
authors: Irungu, J. K.; Munyua, P.; Ochieng, C.; Juma, B.; Amoth, P.; Kuria, F.; Kiiru, J.; Makayotto, L.; Abade, A.; Bulterys, M.; Hunsperger, E.; Emukule, G. O.; Onyango, C.; Samandari, T.; Barr, B. A. T.; Akelo, V.; Weyenga, H.; Munywoki, P. K.; Bigogo, G.; Otieno, N. A.; Kisivuli, J. A.; Ochieng, E.; Nyaga, R.; Hull, N.; Herman-Roloff, A.; Aman, R.
title: Diagnostic accuracy of the Panbio COVID-19 Antigen rapid test device for SARS-CoV-2 detection in Kenya, 2021: A field evaluation
date: 2022-05-25
journal: nan
DOI: 10.1101/2022.05.23.22275439
sha: 9575dd2ff090a9ef3dcd80f8bc309c74db2da151
doc_id: 431384
cord_uid: pujspk2h

Background: Accurate and timely diagnosis is essential in limiting the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. Real-time reverse transcription-polymerase chain reaction (rRT-PCR), the reference standard, requires specialized laboratories, costly reagents, and a long turnaround time. Antigen rapid diagnostic tests (Ag RDTs) provide a feasible alternative to rRT-PCR since they are quick, relatively inexpensive, and do not require a laboratory. The WHO requires that Ag RDTs have a sensitivity [≥]80% and specificity [≥]97%. Methods: This evaluation was conducted at 11 health facilities in Kenya between March and July 2021. We enrolled persons of any age with respiratory symptoms and asymptomatic contacts of confirmed COVID-19 cases. We collected demographic and clinical information and two nasopharyngeal specimens from each participant for Ag RDT testing and rRT-PCR. We calculated the diagnostic performance of the Panbio Ag RDT against the US Centers for Disease Control and Prevention's (CDC) rRT-PCR test. Results: We evaluated the Ag RDT in 2,245 individuals where 551 (24.5%, 95% CI: 22.8-26.3%) tested positive by rRT-PCR. Overall sensitivity of the Ag RDT was 46.6% (95% CI: 42.4-50.9%), specificity 98.5% (95% CI: 97.8-99.0%), PPV 90.8% (95% CI: 86.8-93.9%) and NPV 85.0% (95% CI: 83.4-86.6%). Among symptomatic individuals, sensitivity was 60.6% (95% CI: 54.3-66.7%) and specificity was 98.1% (95% CI: 96.7-99.0%). Among asymptomatic individuals, sensitivity was 34.7% (95% CI 29.3-40.4%) and specificity was 98.7% (95% CI: 97.8-99.3%). In persons with onset of symptoms <5 days (594/876, 67.8%), sensitivity was 67.1% (95% CI: 59.2-74.3%), and 53.3% (95% CI: 40.0-66.3%) among those with onset of symptoms >7 days (157/876, 17.9%). The highest sensitivity was 87.0% (95% CI: 80.9-91.8%) in symptomatic individuals with cycle threshold (Ct) values [≤]30. Conclusion: The overall sensitivity and NPV of the Panbio Ag RDT were much lower than expected. The specificity of the Ag RDT was high and satisfactory; therefore, a positive result may not require confirmation by rRT-PCR. The kit may be useful as a rapid screening tool for only symptomatic patients in high-risk settings with limited access to RT-PCR. A negative result should be interpreted based on clinical and epidemiological information and may require retesting by rRT-PCR.

The COVID-19 pandemic has had major negative socioeconomic effects. Prompt diagnosis of SARS-CoV-2 infection is critical in controlling the pandemic. Real-time reverse transcriptionpolymerase chain reaction (rRT-PCR) remains the reference standard for the diagnosis of SARS-CoV-2 infection. However, rRT-PCR has limitations, including the need for specialized laboratory, equipment, and staff, longer turnaround time (TAT) for clinical decision making and prevention measures, high cost, and erratic supply of reagents to conduct SARS-CoV-2 rRT-PCR due to increased demand (1, 2) . These limitations have increased the demand for more rapid, cheaper, and easy to perform testing methods.

Antigen rapid diagnostic tests (Ag RDTs) increase patients' access to diagnosis of SARS-CoV-2 infection. The World Health Organization (WHO) recommends Ag RDTs with a minimum sensitivity and specificity ≥80% and ≥97%, respectively where rRT-PCR is unavailable or is associated with long TAT (3) .

We evaluated the performance of the Panbio™ Ag RDT compared to the reference standard rRT-PCR. The Panbio™ Ag RDT is authorized in Kenya and received an emergency use license (EUL) from WHO (4) . We also assessed the operational characteristics of the Panbio™ Ag RDT in the field setting.

Methods for more than 15 minutes; traveling together in any kind of conveyance; living in the same household; healthcare-associated exposure, including providing direct care for COVID-19 patients, visiting patients or staying in the same close environment.

Individuals who met the case definition and consented were enrolled and assigned unique identifiers. We collected demographic information, and clinical information via electronic questionnaires on smartphones. Data was stored in a secured cloud server. Clinical information collected included the range and duration of respiratory symptoms, presence of comorbidities, and reported history of exposure to confirmed COVID-19. Two nasopharyngeal swabs were collected by a surveillance officer [nurse, clinical officer, or laboratory technologist] trained to collect respiratory specimens from each of the nostrils a few seconds apart. The first specimen was collected using the swab provided in the Panbio™ Ag RDT kit. The second specimen was collected using a polyester-tipped aluminum shafted swab and immediately placed in viral transport media at +2 to +8 0 C and shipped to the CDC-supported KEMRI laboratories in Nairobi or Kisumu for rRT-PCR testing where long-term storage was done at -80 o C.

Rapid testing was conducted using the Panbio™ Ag RDT (Abbott Rapid Diagnostics, US. Ref.

41FK10) per manufacturer instructions by surveillance officers. The Panbio™ Ag RDT is a lateral flow immunoassay that detects the nucleocapsid (N) protein in nasal and nasopharyngeal specimens (5) . Results were read and documented independently by two surveillance officers within 15 to 20 minutes. Positive, negative, and invalid results were interpreted according to the manufacturer's recommendations. Each reader documented the test results in the electronic . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) Assessing the operational characteristics of the Panbio™ Ag RDT kit

We assessed operational characteristics of the Panbio™ Ag RDT in the field setting including clarity of kit instructions, technical complexity or ease of use, and the ease of interpretation of results via a standardized questionnaire administered to users on site.

According to the manufacturer's instructions, total nucleic acid material was extracted from 200µL of the nasopharyngeal specimen using either the Ambion MagMax Total RNA isolation kit (Thermo Fisher Scientific, Greenville, North Carolina, USA) performed on a semi-automated KingFisher flex machine (Thermo Fisher Scientific) or Standard M Spin-X Viral RNA Extraction Kit (SD biosensor) according to the manufacturer's instructions. We then eluted 60uL of total RNA from the extracted sample. The SARS-CoV-2 rRT-PCR was performed using the TaqPath 1-step Multiplex rRT-PCR master mix (Thermo Fisher Scientific) or using the GoTaq® Probe 1-Step RT-qPCR System (Promega A6120). Both systems are compatible with the rRT-PCR CDC IDT kit. Each of the 15µL reaction volumes contained 5µL of master mix, 8.5µL of nuclease-free water, and 1.5µL of CDC-IDT primer/probe mix for each of the N1, N2, and RNP3 genes separately. The eluate was then amplified using QuantStudio™ 5 Real-Time PCR System, 0.1ml block (Thermo Fisher Scientific), and analyses were done using QuantStudio 3 and 5 Real-Time PCR System Software version 1.5.1 (Thermo Fisher Scientific). SARS-CoV-2 assay targets, N1 and N2 (N1 and N2) were run simultaneously with the human ribonuclease P gene (RNP) control to monitor the quality of the nucleic acid extraction, specimen quality, and . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 25, 2022. ; https://doi.org/10.1101/2022.05.23.22275439 doi: medRxiv preprint presence of reaction inhibitors for assay performance. The positive, negative, and human specimen controls were included in all assays.

When all controls exhibited expected performance, a positive result for SARS-CoV-2 was considered, if all assay amplification curves crossed the threshold line within Ct <40. If SARS-CoV-2 (N1 and N2) assay results were negative, then the test result was reported as negative. If only one target (N1 and N2) were positive, then it was designated as inconclusive, and retesting was conducted. If upon repeating the test, the results remain inconclusive, then the final result was reported as inconclusive. Results were relayed back to the specific sites for patient management.

To evaluate the diagnostic performance of the Panbio™ Ag RDT device, we assumed a prevalence of 10% among symptomatic individuals and 5% in asymptomatic contacts of laboratory-confirmed persons based on current SARS CoV 2 local surveillance data. We assumed a sensitivity of 90% for the Panbio™ Ag RDT with 7.5% margin of error, and after adjusting by 10% for other concerns we anticipated enrolling 770 symptomatic patients and 1,540 close contacts of confirmed cases.

Statistical analysis was conducted using Stata version 15.1 (College Station, Texas). Measures of central tendency and dispersion were calculated for continuous variables such as age, days since onset of symptoms, temperature, and cycle threshold (Ct) values. We calculated frequencies and proportions for categorical variables such as age group, occupation, and Ct value cut-offs. The . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 25, 2022. ; https://doi.org/10.1101/2022.05.23.22275439 doi: medRxiv preprint 95% confidence intervals for categorical variables were calculated using the normal approximation method (Wald's test). We compared proportions between the symptomatic and asymptomatic groups by calculating their Z scores. P-values <0.05 were considered statistically significant.

We calculated the sensitivity, specificity, and positive and negative predictive values for the Panbio™ Ag RDT against rRT-PCR as the reference standard. Stratified analysis of performance was done by age, day's post-onset of symptoms, and Ct values. The measure of agreement between Ag RDT and rRT-PCR results was evaluated using Cohen's Kappa statistic. We . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) 

We enrolled 2,279 participants, where 38.9% had at least one respiratory symptom. The median age was 31.0 years, and over half (60.9%) were male ( Table 1 ). The median duration post-onset of symptoms among symptomatic participants was three days, with the majority (67.8%) presenting within 5 days. Among the 2,277 tested using the Panbio™ Ag RDT device, 12.4% were positive ( Figure 1 ). There were no invalid or indeterminate Ag RDT results. The positivity was significantly higher among symptomatic individuals (18.7%) compared with the asymptomatic (8.4%,) individuals. rRT PCR test results

We tested 2,277 specimens by rRT-PCR, 24.2% tested positive and 1.4% were inconclusive. Positivity (28.7%,) was higher among symptomatic participants compared to asymptomatic is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 25, 2022. asymptomatic individuals (p <0.001). The median Ct value was significantly lower among those with Ag RDT positive results (23.2) compared to negative test results (35.1) (p <0.001) ( Figure   2 ). 

We reviewed 2,245 paired Ag RDT/rRT-PCR records for the performance analysis. The overall sensitivity of the Panbio™ Ag RDT was 46.6% and specificity was 98.5% (Table 2 ). At the observed SARS-CoV-2 prevalence (24.5%) the positive predictive value (PPV) was 90.8% and the negative predictive value (NPV) was 85.0% (Table 2 ; Supplementary Figures 1 & 2) . The overall agreement between the Ag RDT and rRT-PCR results was moderate (k=0.5,), substantial (k=0.7) among the symptomatic, and fair (k=0.4) among the asymptomatic. The performance of the Panbio™ Ag RDT is summarized in Table 2 . and declined to 53.3% among those with onset >7 days. Among the asymptomatic, sensitivity . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 25, 2022 Table 2) .

Evaluation of the user's experience of the Panbio TM Ag RDT A total of 17 surveillance officers comprising 10 (59%) clinicians and 7 (41%) laboratory staff completed the survey. All 17(100%) surveillance officers found the test procedure easy to perform, and the manufacturers' instructions were found to be clear and easy to follow. Four (23.5%) of the surveillance officers noted the difficulty in dispensing a sample onto the test device from the extraction tube when nasopharyngeal specimens were collected from individuals with thick mucus.

We present findings of a field evaluation of the Panbio™ Ag RDT conducted among both symptomatic patients with onset of symptoms up to 14 days and asymptomatic individuals who had contact with individuals with confirmed SARS-CoV-2 infection within two days before detection of the index case and up to 14 days after exposure. Our evaluation was conducted in the third COVID-19 wave in Kenya(7,8).

The overall sensitivity was low (46.6%), detecting slightly less than half of the rRT-PCR confirmed cases, therefore, missing more than half and classifying them as not having SARS-CoV-2 infection when they were indeed infected. The overall sensitivity we report was lower . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The low overall sensitivity observed may be due to the high proportion (61.1%) of asymptomatic participants, and the higher Ct values we observed in these individuals. The lower performance could also be due to the relatively younger (median age 31 years) population we enrolled compared to other studies we reviewed where the median age was higher (39-51.5 years) (11,13). A recent study demonstrated increasing viral load with increasing age (14); higher sensitivities have been observed among individuals with higher viral loads compared to lower ones. The alpha variant and delta variants predominated our evaluation in Kenya (15, 16) . While some literature indicates that variants of concern (17,18) do not reduce the sensitivity and specificity of Ag RDTs, a recent study revealed that sensitivity of the Panbio™ Ag RDT declined significantly in individuals infected with the alpha variant (53.0%) compared to those with non-alpha variants (89.0%) even after adjusting for viral load (p <0.002) (19) . Mutations occurring in the N protein of the Alpha variant may not be detected by the Panbio™ Ag RDT (20) .

The sensitivity among symptomatic individuals (60.6%) was lower than the WHO recommendation of 80% (21) . The sensitivity among symptomatic individuals with onset of symptoms within 7 days was also lower than that reported by the manufacturer on specimens tested on nasopharyngeal specimens (61.6% vs. 91.1%) (5) . This is possibly due to the use of specimens with high viral loads by the manufacturer's study for approval as hypothesized by . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) (25) (26) (27) . We also observed higher viral loads among symptomatic individuals (median Ct = 26.2) compared to asymptomatic individuals (median Ct = 33.0) from our evaluation (p <0.001). Similar findings have also been reported recently (28, 29) . In contrast, some studies found no difference in viral load between the two groups (30,31,) while others found higher viral load among asymptomatic individuals (32,33). Higher sensitivity has been observed among individuals presenting in the early symptomatic phase of their infection (12, 22, (25) (26) (27) (34) (35) (36) . While rRT-PCR testing for SARS-CoV-2 is the reference standard for diagnosis limited access to molecular tests in our setting and provision of results in a clinically relevant timely manner to inform case management limits their clinical utility. Therefore, given reasonable sensitivity, individuals with a negative test in an individual meeting . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The overall specificity we observed (98.5%) was high and was above the WHO recommendation for both symptomatic and asymptomatic individuals and across the other subgroups (21) and was similar to observations in similar evaluations (22, 25, 26, 35, (38) (39) (40) (41) . The Ag RDT had high specificity with less than (1.5%) of the tests (n=26) giving a false-positive result. This small proportion represents individuals who would be falsely classified as infected and unnecessarily be managed as COVID-19 cases or self-isolated and their contacts traced and quarantined. However, we observed over half (53%) of the Ag RDT results gave false-negative results with a higher proportion (65%) among the asymptomatic individuals. This observation emphasizes the recommendation by WHO to re-test symptomatic patients with rRT-PCR when they receive a negative Ag RDT result, especially in settings where SARS-CoV-2 prevalence is ≤ 5%. Further, as a screening tool for testing contacts of confirmed cases, the 65% false negatives among . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 25, 2022. ; https://doi.org/10.1101/2022.05.23.22275439 doi: medRxiv preprint asymptomatic contacts represent the proportion that would falsely be classified as virus-free, not self-isolate, and potentially infect others.

The Panbio™ Ag RDT was found to be easy to use by the majority of the users with a quick TAT for clinical decision making and implementation of preventive measures to contain transmission as observed in the literature (25, 26) . The main challenge we observed in conducting Ag RDT testing using the Panbio™ Ag RDT kits was dispensing the specimen onto the test device in samples collected from individuals with thick mucus. This was, however, solved by ensuring that individuals blew their noses before specimen collection.

Our field evaluation had several strengths. The evaluation was conducted in the real-world setting across multiple sites under the point-of-care conditions which had a large sample size comprising two groups (symptomatic and asymptomatic individuals) thereby representing individuals at any point of the COVID-19 disease spectrum.

Limitation: The number of persons enrolled in the two groups, symptomatic and asymptomatic, may not reflect the true prevalence in the general population as we evaluated the Ag RDT during a period of high COVID-19 prevalence.

The overall sensitivity and PPV of the Panbio™ Ag RDT were much lower than expected.

Sensitivity was acceptable in symptomatic individuals with Ct value ≤30. Specificity of the Ag RDT was high and satisfactory; therefore, a positive result may not require confirmation by rRT-PCR. The kit may be useful as a rapid screening tool for only symptomatic patients in high-risk . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 25, 2022 . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 25, 2022 . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 25, 2022. ; https://doi.org/10.1101/2022.05.23.22275439 doi: medRxiv preprint

Field evaluation of a rapid antigen test (Panbio TM COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres

Overcoming supply shortage for SARS-CoV-2 detection by RT-qPCR

Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays Interim guidance

WHO Emergency Use Listing for In vitro diagnostics (IVDs) Detecting SARS-CoV-2. Geneva [Internet]. World Health Organization (WHO)

TM Test Device COVID-19 Ag Rapid -In vitro diagnostic rapid test for qualitative detection of SARS-CoV-2 antigen (Ag)

The Measurement of Observer Agreement for Categorical Data

Genomic surveillance detects local transmission of the global variants of concern in nine counties in Kenya

A genomics dissection of Kenya's COVID-19 waves: temporal lineage replacements and dominance of imported variants of concern

SARS-CoV-2 rapid diagnostic tests for emerging variants. The Lancet Microbe

SARS-CoV-2 antigen-detecting rapid tests for the delta variant. The Lancet Microbe

Comparison of the Performance of the PanBio COVID-19

B.1.1.7 (Alpha) Variants versus non-B.1.1.7 Variants

SARS-CoV-2 variants with T135I nucleocapsid mutations may affect antigen test performance

SARS-CoV-2 Antigen detecting rapid diagnostic test implementation projects

Evaluation of Panbio rapid antigen test for SARS-CoV-2 in symptomatic patients and their contacts: a multicenter study

Real-life evaluation of a COVID-19 rapid antigen detection test in hospitalized children

Diagnostic accuracy of sars-cov-2 rapid antigen detection testing in symptomatic and asymptomatic children in the clinical setting

The Abbott PanBio WHO emergency use listed, rapid, antigen-detecting point-of-care diagnostic test for SARS-CoV-2-Evaluation of the accuracy and ease-of-use

Evaluation of Abbott Panbio COVID-19 Ag Rapid Test Device External Report

Analytical and clinical performance of the panbio COVID-19 antigen-detecting rapid diagnostic test

The high prevalence of asymptomatic SARS-CoV-2 infection reveals the silent spread of COVID-19

Comparison of Reverse-Transcription Polymerase Chain Reaction Cycle Threshold Values From Respiratory Specimens in Symptomatic and Asymptomatic Children With Severe Acute Respiratory Syndrome Coronavirus 2 Infection

Available from: Supplementary figure 4: The specificity of the Panbio™ Ag RDT by clinical status and days post onset of symptoms

RDT-ve/rRT-PCR -ve 1668

RDT-ve/rRT-PCRve 610

RDT-ve/rRT-PCR -ve 1058, rRT-PCR -ve 1072) <5 days (Ag RDT-ve/rRT-PCR -ve 428, rRT-PCR -ve 436) 5-7 days (Ag RDT-ve/rRT-PCR -ve 87, rRT-PCR -ve 89) >7-14 days (Ag RDT-ve/rRT-PCRve 95, rRT-PCR -ve 97) ≤7 days (Ag RDT-ve/rRT-PCR -ve

>7-14 days (Ag RDT-ve/rRT-PCRve 53, rRT-PCR -ve 54) Cases Days post onset of symptom