key: cord-0887425-jl8228pc
authors: Péré, Hélène; Bouassa, Ralph-Sydney Mboumba; Tonen-Wolyec, Serge; Podglajen, Isabelle; Veyer, David; Bélec, Laurent
title: Analytical performances of five SARS-CoV-2 whole-blood finger-stick IgG-IgM combined antibody rapid tests
date: 2021-01-18
journal: J Virol Methods
DOI: 10.1016/j.jviromet.2021.114067
sha: adbe04ee42ff6e1ad18a42fd86c3a24d7489ced9
doc_id: 887425
cord_uid: jl8228pc

Facing the ongoing pandemic caused by SARS-CoV-2, there is an urgent need for serological assays identifying individuals previously infected by coronavirus disease 2019 (COVID-19), including rapid diagnostic tests (RDTs). We herein compared five new CE-IVD-labeled commercially available SARS-CoV-2 whole-blood finger-stick IgG/IgM combined RDTs, in parallel according to the manufacturers’ instructions, with two serum panels obtained from 48 patients with confirmed COVID-19 (panel I) and from a group of 52 patients randomly selected, for whom serum samples collected before the COVID-19 epidemic (from October 1 to November 30, 2019) were negative for SARS-CoV-2 IgG (panel II). We found a sensitivity of 95.8%, 91.6%, 92.3%, 97.9% and 91.4%, and a specificity of 98.1%, 86.5%, 100%, 98.1% and 84.6%, for BIOSYNEX COVID-19 BSS (IgG/IgM) (Biosynex Swiss SA, Freiburg, Switzerland), Humasis COVID-19 IgG/IgM Test (Humasis Co., Ltd., Gyneonggi, Republic of Korea), LYHER COVID-19 IgM/IgG Rapid Test (Medakit Ltd, Hong Kong, China), SIENNA™ COVID-19 (IgG/IgM) Rapid Test Cassette (Salofa Oy, Salo, Finland) and NG-BIOTECH COVID-19 (IgG/IgM) (NG-Biotech, Guipry, France), respectively. Commercially available SARS-CoV-2 IgG/IgM combined RDTs have a sufficient sensitivity for identifying individuals with past SARS-CoV-2 infection, but some RDTs may lack of specificity, with risk of false positivity mainly for the IgM band.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a novel coronavirus that causes Coronavirus Disease 2019 , started in Wuhan province, China, in December 2019. It was declared by the World Health Organization (WHO) as global pandemic on March 11 th , 2020 (WHO, 2020a). SARS-CoV-2 infection has caused a worldwide viral pandemic, with >630,781 deaths and >18,261,656 cases reported internationally, with France reporting 187,919 cases and 30,265 deaths, as of 03 rd August 2020 (Worldometer, 2020) .

Control of the outbreak in both community and hospital setting has mainly relied on the availability of highly sensitive and specific nucleic acid amplification-based molecular testing for SARS-CoV-2 Wölfel et al., 2020) . Furthermore, it was demonstrated that serological testing looking for specific SARS-CoV-2 IgG and/or IgM antibodies could be useful for confirming the diagnosis and care of COVID-19 (Amanat et al., 2020; Long et al., 2020; Petherick, 2020) . On March 2 nd , 2020, the WHO recommended serological testing in addition to molecular diagnosis, for investigating on-going outbreaks as well as for the diagnosis of strongly suspected patients of SARS-CoV-2 infection with a negative molecular test for SARS-CoV-2 RNA (WHO, 2020b) . Otherwise, antibody tests for SARS-CoV-2 could constitute one of the keys to fight the SARS-CoV-2 epidemic, in particular to surpass the deconfinement period (Petherick et al., 2020) . Seropositivity to SARS-CoV-2 antigens would also allow the identification of previously infected individuals, including asymptomatic patients, a priori considered to be healed and protected against new reinfection (Petherick et al., 2020) , although this use remains controversial (Belec et al., 2020) .

Recently, rapid diagnostic tests (RDTs) for IgG/IgM antibodies produced during COVID-19 have been developed (Abduljalil, 2020; La Marca et al., 2020) , and reports have shown that COVID-19 IgG/IgM lateral flow immunoassays may be a reliable tool to diagnose SARS-CoV-2 infection from 14 days following the onset of symptoms (Candel et al., 2020; Charlton et al., 2020; Delliere et al., 2020; Montesinos et al., 2020; Nicol et al., 2020; Pérez-Garcia et al., 2020) .

In some countries, rapid diagnostic testing for COVID-19 has been incorporated into the current local guidelines for testing asymptomatic contacts of positive cases, at day 13 of home surveillance (Zainol Rashid et al., 2020) . These easy to use IgG/IgM combined test kits allow rapid screening with capillary blood sample. The tests are simple, qualitative, visually interpretable, and give a result within 10 to 15 minutes. A positive serology allows determining whether a person has already been infected by SARS-CoV-2. Serologic tests will be needed to assess the immune response of vaccine candidates and to map levels of immunity in communities. These RDTs are particularly interesting for low resource settings, such as at patients' bedside or at any other locations where laboratory facilities are lacking.

The analytical performances of RDTs for both SARS-CoV-2-specific IgG and IgM detection should be investigated before use on local serum panels, as recommended for other rapid tests, such as HIV RDTs (WHO, 2012; WHO, 2015) . We herein investigated and compared the analytical performances of 5 CE IVD-labeled immunochromatographic RDTs for SARS-CoV-2-specific IgG and IgM detection using serum J o u r n a l P r e -p r o o f panels from patients with COVID-19 hospitalized during the SARS-CoV-2 infection outbreak in last March 2020 and from patients hospitalized in Paris, France, collected prior the 2019 COVID-19 outbreak.

Two groups of patients were included in the study.

A panel of 48 serum samples was prospectively constituted during the epidemic period of SARS-CoV-2 infection in France between March and April 2020 from inpatients (30 males; 18 females, mean age, 62.3 years) of Hôpital Européen Georges Pompidou, Paris, France, suffering from COVID-19 with positive detection of SARS-CoV-2 from a nasal swab by molecular diagnosis (Allplex™ 2019-nCoV Assay, Seegene, Seoul, Korea). Samples were collected at least 4 weeks after the onset of clinical signs suggestive of COVID-19 (mean 36.5 days). All sera were furthermore found positive for SARS-CoV-2 IgG by CE-IVD Abbott SARS-CoV-2 IgG assay (Abbott GmbH, Rungis, France) detecting IgG against SARS-CoV-2 nucleoprotein used on Architect analyzer (Abbott Architect™ i2000), according to manufacturer's instructions. This panel was considered as "true positive sera".

A panel of 52 serum samples randomly selected from patients (37 males; 15 females, mean age, 59.4 years) who had a serum sample taken for other serologic studies, from September 1 st to November 30 th , 2019 (Prior to the first cases of COVID-19 were reported in France). All sera were negative for Abbott SARS-CoV-2 IgG reference serology. Thirteen had previous pulmonary infection with endemic coronavirus [n=12; 229E (n=2), OC43 (n=4), NL63 (n=6)] or metapneumovirus (n=1) detected by the real-time Allplex™ Respiratory Panel RT-PCR assay (Seegene). Eighteen showed positive for malaria serology. Nineteen had IgG against cytomegalovirus (CMV). This panel was considered as "true negative sera".

The following commercially available CE IVD-labeled finger-stick whole-blood RDTs for IgG and IgM antibodies to SARS-CoV-2 detection were used in the study:

 

All data were entered into an Excel file and analyzed on SPSS 20.0 (Chicago, IL). Test results in group 1 were considered as "true positive" (TP) in case of positive test result and "false negative" (FN) in case of negative test result. Test results in group 2 were considered as "true negative" (TN) in case of negative test results and "false positive" (FP) in case of positive test result. The sensitivity and specificity were calculated for each serologic test using the results from group 1 and group 2 patients, respectively.

First, the percent agreement was estimated to measure crude agreement between the test results and the expected results without taking into account the agreement due solely to chance. Next, the kappa concordance, using the Cohen's κ coefficient formula (Cohen, 1960) , was calculated to measure agreement by taking into account the agreement due to chance. Thus, the degree of agreement was determined as ranked J o u r n a l P r e -p r o o f by Landlis and Koch (Landlis & Koch, 1997) . The accuracy of each serologic test was estimated by Youden's J index (J = sensitivity + specificity − 1) (Youden, 1950) . The results were presented as a 95% confidence interval (CI) using the Wilson score bounds (Newcombe, 1998) .

Although there is no international consensus on the minimum sensitivity and specificity of RDTs for COVID-19 serological diagnosis, the French technical agency for medical practice and biological tests, the so-called Haute Autorité de Santé (HAS, Saint-Denis, France), has proposed on April 16 th , 2020, that analytical performances of such rapid tests should demonstrate minimum clinical sensitivity of 90% (or even 95%) and clinical specificity of 98% (Haute Autorité de Santé, 2020).

Our non-interventional study was carried out in accordance with the Declaration of Helsinki with no additional sampling to usual procedures. Serum sample specimens were obtained in France only for standard diagnostic following medical prescriptions and care. Under these conditions, the study was exempt from informed consent application, according to the French public health code (Code de la Santé Publique, article L 1121-1.1; https://www.legifrance.gouv.fr/). Data analyses were carried out using anonymized database.

The results of IgG and IgM reactivity of the 5 study RDTs for SARS-CoV-2-specific antibodies detection by using the per-epidemic serum panel positive for SARS-CoV-2 IgG and the pre-epidemic serum panel are depicted in Table 1 .

All false positive results were due to weak IgM reactivity at 1 cross. There was no IgG reactivity in our series. Interestingly, the IgM reactivity was associated with other infections for Humasis COVID-19 IgG/IgM Antibody Test [5 of 7 (71.4%): seropositivity for CMV (n=2) and malaria (n=1), infection with OC43 coronavirus (n=1) and OC43 coronavirus and metapneumovirus (n=1)] and NG-BIOTECH COVID-19 (IgG/IgM) [5 of 8 (62.5%): seropositivity for CMV (n=1) and malaria (n=2), infection with NL63 coronavirus (n=1) and OC43 coronavirus and metapneumovirus (n=1)], respectively.

Analytical performances are shown in the Table 1 .

We found a sensitivity of 95.8%, 91.6%, 92.3%, 97.9% and 91.4%, and a specificity of 98.1%, 86.5%, 100%, 98.1% and 84.6%, for BIOSYNEX COVID-19 BSS (IgG/IgM), Humasis COVID-19 IgG/IgM Test, LYHER COVID-19 IgM/IgG Rapid Test, SIENNA™ COVID-19 (IgG/IgM) Rapid Test Cassette and NG-BIOTECH COVID-19 (IgG/IgM), respectively.

The concordance of the 5 study RDTs were excellent: almost perfect, respectively, for 3 tests, and good and substantial, respectively, for the 2 remaining RDTs. The accuracy to diagnose a "true positive" or a "true negative" serum showed similar distribution, with 3 study RDTs with excellent accuracy, and the 2 remaining with only acceptable accuracy. Finally, the 3 out 5 study RDTs fulfilling the acceptance criteria of the Haute Autorité de Santé, showed the best analytical performances, including sensitivity, specificity, agreement, concordance and accuracy. The CE IVD Humasis COVID-19 IgG/IgM Antibody Test and NG-BIOTECH COVID-19 (IgG/IgM), approved by the French Ministry of Health, did not respect the Haute Autorité de Santé criteria in our hands.

In our study, we showed that rapid CE IVD-marked RDTs for the detection of circulating antibodies of the IgG and IgM isotypes against SARS-CoV-2 could harbor weak positive reactivity with sera collected from European patients before the 2019 outbreak of COVID-19, always for the IgM isotype, leading to possible final misinterpretations, i.e. false-positive test results. Taken together, our observations point the risk of J o u r n a l P r e -p r o o f false positive reactivity when using currently available RDTs for SARS-CoV-2 serological screening, especially for the IgM band, even if the test is CE IVD-labeled and approved by national health authorities. These findings have important implications for the choice of the best COVID-19 serological assays. Finally, the need for validating the analytical performances of COVID-19 RDTs using local serum panels well documented and mastered by the biologist should be underlined.

COVID-19 infection can be detected indirectly by measuring the host humoral immune response to SARS-CoV-2 infection, SARS-CoV-2-specific IgM indicating acute or recent infection and SARS-CoV-2-specific IgG on-going or past-infection (Sethurama et al., 2020) . Positive reactivity, particularly for the IgM band, and resulting false positive test results were variably observed for the 5 study COVID-19 RDTs used with pre-epidemic sera. These observations are reminiscent of the high rates of unspecific serological reactivity with immunochromatographic RDTs that have been previously reported for various infectious diseases, such as HIV (Aghokeng et al., 2009; Jenabian et al., 2017) or syphilis (Mbopi Keou et al., 2019) . In our study, about 50% of patients exhibited positive reactivities and evidence of recent infections with common cold coronaviruses or other respiratory virus, or positive results for HIV, HBV, CMV, malaria, or HCV serologies. These associations allow us to hypothesize that a variety of mechanisms related to infectious diseases may be involved to account for false positive IgM results, such as crossreacting antibodies against common coronaviruses (Woo et al., 2004) , or disturbances affecting the B cell-driven immunity during infectious diseases (Mori et al., 1987; Bouvet & Dighiero, 1998; Klarkowski et al., 2014; Mbopi Keou et al., 2014) . However, in the remaining study positive reactivities, no obvious co-infectious could be found, suggesting that other causes of unspecific RDT results could be also involved such as autoimmunity, concurrent infection by other pathogens, interfering substances such as rheumatoid factor and other yet unknown factors (Landry, 2016) . It is well recognized that most IgM based serological assays for several infectious diseases diagnosis suffer from higher false positive rates relative to IgG-based assays (Landry, 2016) , as in our study COVID-19 RDT.

Diagnostic test performances must be examined within the specific settings where the tests will be used, and this may be relevant to account for possible differences due to environmental and racial factors (Mbopi Kéou et al., 2019) .

RDTs for detection of SARS-CoV-2-specific antibodies have been widely developed and are of variable quality. Currently, there is greater than 200 RDTs available for the detection of antibodies against SARS-CoV-2 (Covid-19 -Foundation for Innovative New Diagnostics; available at: https://www.finddx.org/). Many manufacturers do not reveal the nature of antigens used. These tests are purely qualitative in nature and can only indicate the presence or absence of SARS-CoV-2 antibodies. In order to make the testing more available, the principal regulatory agencies such as the Food and Drug Administrations (FDA, US) and the European Center for Disease Prevention and Control have removed demanding requirements for agency review of SARS-CoV-2 assays, providing a double-edged sword, with test not properly vetted and with highly variable performances. All the serological tests in our study have the CE IVD mark and four of them have been approved by the French Ministry of Health, i.e. they have theoretical clinical sensitivity of at least 90% and specificity of at least 98% (Haute Autorité de Santé, 2020). However, even for compliant CE-marked RDTs, their performance may vary in the routine testing laboratory in comparison with the performance study of the manufacturer done for the purposes of CE-marking. Among our 5 study RDTs for COVID-19 serology, Humasis COVID-19 IgG/IgM Test and NG-Test ® IgG-IgM COVID-19 showed poor analytical performances assessed by high false positive test results rates with the European (13.4% and 15.4%, respectively) pre-epidemic sera, in accordance with the restricted use of both tests to IgG, not IgM by J o u r n a l P r e -p r o o f the French Ministry of Health. Higher specificity at 95.3% was previously reported for the NG-Test ® IgG-IgM COVID-19, but the difficulty for reading the IgM band of this test was underlined (Nicol et al., 2020) .

Among the 5 study RDTs, the tests LYHER COVID-19 IgM/IgG Rapid Test, BIOSYNEX ® COVID-19 BSS (IgG/IgM) and SIENNA™ COVID-19 (IgG/IgM) Rapid Test Cassette showed the lowest rates of false positive test results with European pre-epidemic sera (0%, 1.9%, 1.9%, respectively). Cautionary opinion on currently available SARS-CoV-2 serological assay was previously expressed (Farnsworth & Anderson, 2020) , despite the excitement of the lay public emphasizing that the deployment of insufficiently validated assays, particularly inaccurate ones, could have negative consequences for public health.

Our study has some limitations. In particular, the number of blood samples is limited. Thus, the number of negative specimens to be analyzed should be sufficiently high, especially if the prevalence of COVID-19 in the community is low and if the risk of false positive results is low, as for example for specific antibodies of the IgG isotype (Farnsworth & Anderson, 2020) . In addition, the positive sera from group 1 were collected from patients at least 4 weeks after the onset of clinical signs, late in the kinetic infection and probably containing only SARS-CoV-2specific IgG. Thus, we can consider that the study results refer exclusively to a specific phase of the infection (at least 4 weeks after the beginning of SARSD-CoV-2 infection), not allowing sufficient evaluation of the analytical parameters of the IgM band.

In practice, clinical validation of the diagnostic performance of RDTs for COVID-19 serology in real-life should be always carried out in a sufficiently large number of target population subjects before introducing them into the routine diagnosis, as strongly recommended by the European Centre for Disease Prevention and Control (ECDC 2020). Our study in a hospital setting confirms the need for COVID-19 RDTs to undergo validation on the field in the specific environments in which they will be deployed, with local serum panel corresponding to given environments and human habitats.

In conclusion, the immunochromatographic RDTs for SARS-CoV-2 serological screening are simple, cheap and fast. They do not require qualified personnel for interpretation and could be done in hospitals as well as primary care facilities. However, in view of the great profusion of COVID-19 RDTs on the market, it is important to evaluate them locally before use, in order to confirm their analytical performances, independently of those asserted by the manufacturer.

No grant was received from the test manufacturers.

The authors of this paper have no financial or personal relationship with other people or organizations that could inappropriately influence or bias the content of this paper.

IP, HP, DV and LB have conceived and designed the research; RSMB performed the experiments; STW performed statistical analyses; HP, DV, LB and RSMB analyzed the results and drafted the manuscript.

The authors report no conflicts of interest. The authors alone are responsible for the content and the writing of the paper.

Laboratory diagnosis of SARS-CoV-2: available approaches and limitations

Inaccurate diagnosis of HIV-1 group M and O is a key challenge for ongoing universal access to antiretroviral treatment and HIV prevention in Cameroon

A serological assay to detect SARS-CoV-2 seroconversion in humans

Potential pitfalls of routine SARS-CoV-2 serology for mass screening

From natural polyreactive autoantibodies to à la carte monoreactive antibodies to infectious agents: is it a small world after all?

Utility of lateral flow tests in SARS-CoV-2 infection monitorization

Evaluation of six commercial mid to high volume antibody and six point of care lateral flow assays for detection of SARS-CoV-2 antibodies

A coefficient of agreement for nominal scales

Evaluation of COVID-19 IgG/IgM Rapid Test from Orient Gene Biotech

An overview of the rapid test situation for COVID-19 diagnosis in the EU/EEA

SARS-CoV-2 Serology: Much Hype, Little Data

Cahier des charges définissant les modalités d'évaluation des performances des tests sérologiques détectant les anticorps dirigés contre le SARS-CoV-2

Potential for False-Positive Results with Serological Assays for HIV in Central Africa: Implications for the HIV Serodiagnosis Algorithm According to the 2015 Consolidated WHO Recommendations for Resource-Constrained Countries

Causes of false-positive HIV rapid diagnostic test results

Testing for SARS-CoV-2 (COVID-19): a systematic review and clinical guide to molecular and serological in-vitro diagnostic assays

The measurement of observer agreement for categorical data

Immunoglobulin M for Acute Infection: True or False? Clin

Association of inconclusive sera for human immunodeficiency virus infection with malaria and Epstein-Barr virus infection in Central Africa

Differential influence of race and environment on indeterminate reactivities to non-treponemal and treponemal antigens by immuno-chromatographic dual syphilis rapid test

Evaluation of two automated and three rapid lateral flow immunoassays for the detection of anti-SARS-CoV-2 antibodies

Polyclonal B-cell activation and autoantibody formation during the course of mosquito-transmitted Plasmodium berghei infection in mice

Two-sided confidence intervals for the single proportion: comparison of 362 seven methods

Assessment of SARS-CoV-2 serological tests for the diagnosis of COVID-19 through the evaluation of three immunoassays: Two automated immunoassays (Euroimmun and Abbott) and one rapid lateral flow immunoassay (NG Biotech)

Nasal Swab Sampling for SARS-CoV-2: a Convenient Alternative in Times of Nasopharyngeal Swab Shortage

Alltest rapid lateral flow immunoassays is reliable in diagnosing SARS-CoV-2 infection from 14 days after symptom onset: A prospective single-center study

Interpreting Diagnostic Tests for SARS-CoV-2

Virological assessment of hospitalized patients with COVID-2019

False-positive results in a recombinant severe acute respiratory syndrome-associated coronavirus (SARS-CoV) nucleocapsid enzyme-linked immunosorbent assay due to HCoV-OC43 and HCoV-229E rectified by Western blotting with recombinant SARS-CoV spike polypeptide

Service delivery approaches to HIV testing and counselling: A strategic policy framework

Consolidated guidelines on HIV testing services

Coronavirus disease (COVID-19) outbreak

Laboratory testing for coronavirus disease 2019 (COVID-19) in suspected human cases: interim guidance

Index for rating diagnostic tests

Diagnostic Performance of COVID-19 Serology Assays

The authors are grateful to Céline Pointeau from Service de Microbiologie, Hôpital Européen Georges Pompidou, Assistance Publique-Hôpitaux de Paris, Paris, France, Biosynex, Strasbourg, France, Medequip Integrated Services, Horbourg-Wihr, France, and the Agence Régionale de Santé Île-de-France, Paris, France, for providing for the tests for the study. Dr Serge Tonen-Wolyec was recipient of ERASMUS program between the University of Kisangani, Democratic Republic of the Congo, and the University of Liège, Belgium. We authors thanks Dr Sarah Grech, University of McGill, Montréal, Canada, for the editing of the English of our manuscript.

No a The availability of the LYHER COVID-19 IgM/IgG Rapid Test was limited and 1 out of 2 sera were randomly selected to be tested by this test; the NG-BIOTECH COVID-19 (IgG/IgM) was tested with only 47 sera of group 1, because the quantity of 1 serum of this group was insufficient; * 95% confidence intervals in brackets were calculated using the Wilson score bounds;