key: cord-0913603-el2gadva authors: Coyle, P. V.; El Kahlout, R. A.; Dargham, S. R.; Chemaitelly, H.; Kacem, M. A. B. H.; Al-Mawlawi, N. H. A.; Gillani, I.; Younes, N.; Al Kanaani, Z.; Al Khal, A.; Al Kuwari, E.; Jeremijenko, A. M.; Kaleeckal, A. H.; Latif, A. N.; Shaik, R. M.; Abdul Rahim, H. F.; Nasrallah, G. K.; Yassine, H. M.; Al Kuwari, M. G.; Al Romaihi, H. E.; Tang, P. M.; Bertollini, R.; Al-Thani, M. H.; Abu-Raddad, L. J. title: Assessing the performance of a serological point-of-care test in measuring detectable antibodies against SARS-CoV-2 date: 2021-02-06 journal: nan DOI: 10.1101/2021.02.04.21251126 sha: a258e50f1a8b0ad4251c9992b909eaf1bd1cea47 doc_id: 913603 cord_uid: el2gadva Objective: To investigate the performance of a rapid point-of-care antibody test, the BioMedomics COVID-19 IgM/IgG Rapid Test, in comparison with a high-quality, validated, laboratory-based platform, the Roche Elecsys Anti-SARS-CoV-2 assay. Methods: Serological testing was conducted on 708 individuals. Concordance metrics were estimated. Logistic regression was used to assess associations with seropositivity. Results: SARS-CoV-2 seroprevalence was 63.4% (449/708; 95% CI 59.8%-66.9%) using the BioMedomics assay and 71.9% (509/708; 95% CI 68.5%-75.1%) using the Elecsys assay. There were 62 discordant results between the two assays. One specimen was seropositive in the BioMedomics assay, but seronegative in the Elecsys assay, while 61 specimens were seropositive in the Elecsys assay, but seronegative in the BioMedomics assay. Positive, negative, and overall percent agreements between the two assays were 88.0% (95% CI 84.9%-90.6%), 99.5% (95% CI 97.2%-99.9%), and 91.2% (95% CI 88.9%-93.1%), respectively, with a Cohen kappa of 0.80 (95% CI 0.77-0.83), indicating excellent agreement. Excluding specimens with lower antibody titers, the agreement improved with positive, negative, and overall percent concordance of 91.2% (95% CI 88.2%-93.6%), 99.5% (95% CI 97.2%-99.9%), and 93.9% (95% CI 91.7%-95.5%), respectively, and a Cohen kappa of 0.87 (95% CI 0.84-0.89). Logistic regression confirmed better agreement with higher antibody titers. Conclusion: The BioMedomics COVID-19 IgM/IgG Rapid Test demonstrated excellent performance in measuring detectable antibodies against SARS-CoV-2, supporting the utility of such rapid point-of-care serological testing to guide the public health responses and possible vaccine prioritization. Caused by the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), coronavirus disease 2019 (COVID-19) continues to present a global challenge, causing severe health, social, and economic burdens [1, 2] . Qatar experienced a large SARS-CoV-2 epidemic with a high rate of laboratory-confirmed infections at >60,000 infections per million population (6%) [3] [4] [5] . As part of the national response, public health authorities expanded serological testing for SARS-CoV-2 antibodies, for both healthcare and research purposes [6] [7] [8] [9] . Moreover, antibody status is being deliberated as one of the criteria for COVID-19 vaccine prioritization [10] . To achieve more efficient, cost-effective, and widescale serological testing, the objective of this study was to investigate the performance of a rapid point-of-care antibody test, the BioMedomics COVID-19 IgM/IgG Rapid Test [11] , in comparison with a high-quality, validated, laboratorybased assay, the Roche Elecsys Anti SARS-CoV-2 platform [12, 13] , one of the most extensively used and investigated commercial platforms, having a specificity ≥99.8% [14, 15] and a sensitivity ≥89% [3, 12, 15] . Samples included 708 residual blood serum specimens that were collected and stored between May 30 and November 18, 2020 from individuals receiving routine or other clinical care at Hamad Medical Corporation (HMC), the main provider of healthcare in Qatar, and the nationally-designated provider for COVID-19 healthcare needs. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted February 6, 2021. ; https://doi.org/10.1101/2021.02.04.21251126 doi: medRxiv preprint Serological testing was performed using the Roche Elecsys Anti-SARS-CoV-2 (Roche, Switzerland) assay, a fully-automated electrochemiluminescent immunoassay [13] , and the BioMedomics COVID-19 IgM/IgG Rapid Test (BioMedomics, Inc., United States of America), a lateral flow immunochromatographic assay [11] . The Roche Elecsys Anti-SARS-CoV-2 assay (hereafter "Elecsys") uses a recombinant protein representing the nucleocapsid (N) antigen for determination of antibodies against SARS-CoV-2 [13] . Qualitative anti-SARS-CoV-2 results were generated following the manufacturer's instructions (reactive for optical density (proxy for antibody titer [15] ) cutoff index ≥1.0 vs. nonreactive for cutoff index <1.0) [13] . The BioMedomics COVID-19 IgM/IgG Rapid Test (hereafter "BioMedomics") is a lateral flow immunoassay that contains a colloidal, gold-labeled, recombinant coronavirus antigen and a quality control antibody colloidal gold marker, two detection lines (IgG and IgM lines), and one quality control line (C) fixed on a nitrocellulose membrane [11] . Qualitative anti-SARS-CoV-2 results were generated by reading the detection line(s) [11] . Results of serological testing were subsequently linked to the national centralized SARS-CoV-2 polymerase chain reaction (PCR) testing and hospitalization database that includes records for all PCR testing and COVID-19 hospitalizations in Qatar since the start of the epidemic [16] . The database also includes the severity classification of hospitalized cases based on individual chart reviews completed by trained medical personnel using the World Health Organization (WHO) criteria [17] . Cross-tabulations of serological testing results were conducted using the Elecsys assay as the reference standard. Concordance metrics were estimated and included the positive, negative, and overall percent agreements. In addition, Cohen's kappa statistic was estimated to measure the classified as mild, 19 were classified as moderate, 14 were classified as severe, and 7 were classified as critical (Table S1 of SI) . For all other individuals no severity classification was conducted, due to absence of serious symptoms to require hospitalization and severity assessment, and thus the infection can be assumed to be asymptomatic or mild. No COVID-19 deaths were reported among study participants. The positive, negative, and overall percent agreements between the two assays were estimated at 88.0% (95% CI 84.9%-90.6%), 99.5% (95% CI 97.2%-99.9%), and 91.2% (95% CI 88.9%-93.1%), respectively (Table 1 ). Cohen's kappa statistic was estimated at 0.80 (95% CI 0.77-0.83) indicating excellent agreement between the two assays ( Table 1) . Including only specimens with higher antibody titers in the comparison (i.e., excluding specimens with low Elecsys optical density values <10), the positive, negative, and overall percent agreements between the two assays were estimated at 91.2% (95% CI 88.2%-93.6%), 99.5% (95% CI 97.2%-99.9%), and 93.9% (95% CI 91.7%-95.5%), respectively (Table 1) . Cohen's kappa statistic was estimated at 0.87 (95% CI 0.84-0.89) indicating excellent and superior agreement between the two assays compared to the result for the full sample ( Table 1) . The cutoff at Elecsys optical density value of 10 (to indicate lower antibody titers) was informed by the distribution of the optical density values (Figure 1 ). Including only specimens with low antibody titers in the comparison (i.e. excluding specimens with Elecsys optical density values ≥10), positive, negative, and overall percent agreements between the two assays were estimated at 72.4% (95% CI 62.2%-80.7%), 99.5% (95% CI 97.2%-99.9%), and 91.3% (95% CI 87.4%-94.0%), respectively (Table 1) . Cohen's kappa statistic was estimated at 0.78 (95% CI 0.72-0.83) indicating excellent agreement between the All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted February 6, 2021. ; https://doi.org/10.1101/2021.02.04.21251126 doi: medRxiv preprint two assays, but still an inferior agreement compared to the result for the subsample of higher antibody titers, and for the full sample (Table 1) . Multivariable logistic regression identified significant associations between seropositivity using the BioMedomics assay and both the Elecsys optical density value and the PCR Ct value or its absence (that is no prior PCR-confirmed infection), but no association with the severity of the infection ( Table 2 ). The adjusted odds ratio (aOR) of seropositivity was 5.06 (95% CI 2.72-9.39, p<0.001) for those with higher antibody titers (Elecsys optical density values ≥10) compared to those with lower antibody titers (Elecsys optical density values <10), reflecting much higher agreement between the two measures for specimens with higher antibody titers. The aOR of seropositivity was 0.00 (95% CI 0.00-0.03, p<0.001) for those with a negative result in the Elecsys assay, indicating a very low likelihood of being negative in the Elecsys assay, but positive in the BioMedomics assay. The aOR of seropositivity was 0.30 (95% CI 0.11-0.81, p=0.018) for those with a Ct value ≥30 compared to those with a Ct value <30, possibly because of some PCR false positivity measures for those with higher Ct values. The aOR of seropositivity was 0.28 (95% CI 0.14-0.57, p=0.001) for those with no Ct value (that is no prior PCR-confirmed infection), reflecting a low likelihood of having been exposed to the infection if they were never diagnosed with the infection. The above results document excellent performance of the BioMedomics rapid test in measuring detectable antibodies against SARS-CoV-2. An additional benefit is to help identify those with a PCR false positive result. The performance of this point-of-care test was also as expected for a All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted February 6, 2021. ; https://doi.org/10.1101/2021.02.04.21251126 doi: medRxiv preprint rapid test: it was optimal when antibody titers were high, but less optimal when they were low. Only one test result could have been a false positive, suggesting very high specificity for this rapid test. The specimen was seronegative in the Elecsys assay and there was no record of a PCR-positive test for this individual. These findings demonstrate the utility of this assay to assess past SARS-CoV-2 infection and potential seropositivity as a marker of immunity against infection, particularly considering the growing evidence indicating that natural infection elicits strong protection against reinfection that lasts for at least a few months [20] [21] [22] . These findings also support the concept of using rapid antibody testing for more efficient, cost-effective, and widescale serological testing to guide vaccine prioritization, or possible issuance of immunity passports. This study has a limitation. The performance of the rapid assay was compared to a laboratorybased assay, the Roche Elecsys Anti SARS-CoV-2 platform (and to PCR testing), but not to a gold standard test of seropositivity, as such a test was not available. Having so said, the Elecsys assay is one of the most extensively used and investigated commercial platforms, having a specificity ≥99.8% [13] [14] [15] and a sensitivity ≥89% [3, 12, 15] . Therefore, it is not likely that this limitation could have affected the findings. In conclusion, the BioMedomics rapid point-of-care test demonstrated excellent performance in measuring detectable antibodies against SARS-CoV-2, with better performance for specimens with higher antibody titers, demonstrating the utility of such assays in mass expansion of serological testing to guide public health responses and possible vaccine prioritization. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The authors are grateful for support provided by Hamad Medical Corporation, the Ministry of Public Health, and the Biomedical Research Program and the Biostatistics, Epidemiology, and All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Author contributions: PVC and LJA conceived and designed the study. PVC led the laboratory testing. SRD conducted the statistical analyses and co-wrote the first draft of the manuscript. HC contributed to the statistical analyses. LJA led the data analyses and co-wrote the first draft of the manuscript. All authors contributed to data collection and acquisition, database development, discussions and interpretation of the results, and to the writing of the manuscript. All authors have read and approved the final manuscript. None. All relevant data are available in the main text and supplementary material. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted February 6, 2021. ; https://doi.org/10.1101/2021.02.04.21251126 doi: medRxiv preprint Table Table 1 . Concordance metrics between two SARS-CoV-2 antibody assays: the BioMedomics COVID-19 IgM/IgG Rapid Test and the Roche Elecsys Anti-SARS-CoV-2 including A) all negative and positive specimens, B) negative specimens and specimens with higher antibody titers (excluding specimens with Elecsys optical density values <10), and C) negative specimens and specimens with lower antibody titers (excluding specimens with Elecsys optical density values ≥10). Overall percent agreement . For individuals where no severity classification was conducted, due to absence of serious symptoms requiring hospitalization and severity assessment, infection was assumed to be asymptomatic or mild. All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted February 6, 2021. ; https://doi.org/10.1101/2021.02.04.21251126 doi: medRxiv preprint Supplementary Information Table S1 . Characteristics of all specimens tested using the BioMedomics COVID-19 IgM/IgG Rapid Test and the Roche Elecsys Anti SARS-CoV-2 assay. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Ct-cycle threshold; N/R-not reported; N/A-not applicable; PCR-polymerase chain reaction. *BioMedomics COVID-19 IgM/IgG Rapid Test assay results were generated by reading the marked red detection line(s) [1] . **Roche Elecsys Anti SARS-CoV-2 assay results were generated using reactive for optical density cutoff index ≥1.0 vs. non-reactive for cutoff index <1.0 [2] . # Specimen number ordered by descending optical density value (antibody titers) of the Roche Elecsys Anti SARS-CoV-2 assay. $ Severity per WHO classification [3] . If N/A, no severity classification was conducted due to absence of serious symptoms to require hospitalization and severity assessment. How two tests can help contain COVID-19 and revive the economy The socio-economic implications of the coronavirus pandemic (COVID-19): A review Characterizing the Qatar advanced-phase SARS-CoV-2 epidemic Mathematical modeling of the SARS-CoV-2 epidemic in Qatar and its impact on the national response to Epidemiological investigation of the first 5685 cases of SARS-CoV-2 infection in Qatar Seroprevalence of SARS-CoV-2 infection in the craft and manual worker population of Qatar National healthcare serological testing in the State of Qatar Evidence for and level of herd immunity against SARS-CoV-2 infection: the ten-community study Are commercial antibody assays substantially underestimating SARS-CoV-2 ever infection? An analysis on a population-based sample in a high exposure setting Epidemiological impact of prioritizing SARS-CoV-2 vaccination by antibody status: Mathematical modeling analyses COVID-19 IgM/IgG Rapid Test Independent side-byside validation and comparison of four serological platforms for SARS-CoV-2 antibody testing. The Journal of Infectious Diseases Roche's COVID-19 antibody test receives FDA Emergency Use Authorization and is available in markets accepting the CE mark Evaluation of Roche Elecsys AntiSARS-CoV-2 serology assay for the detection of anti-SARS-CoV-2 antibodies 1598/Evaluation_of_Roche_Elecsys_anti_SARS_CoV_2_PHE_200610_v8.1_FINAL.pdf. Accessed on Multi-center nationwide comparison of seven serology assays reveals a SARS-CoV-2 non-responding seronegative References 1 COVID-19 IgM/IgG Rapid Test Roche's COVID-19 antibody test receives FDA Emergency Use Authorization and is available in markets accepting the CE mark Clinical management of COVID-19