key: cord-1030452-c8lv1ze4
authors: Andrey, Diego O.; Cohen, Patrick; Meyer, Benjamin; Torriani, Giulia; Yerly, Sabine; Mazza, Lena; Calame, Adrien; Arm‐Vernez, Isabelle; Guessous, Idris; Stringhini, Silvia; Roux‐Lombard, Pascale; Fontao, Lionel; Agoritsas, Thomas; Stirnemann, Jerôme; Reny, Jean‐Luc; Siegrist, Claire‐Anne; Eckerle, Isabella; Kaiser, Laurent; Vuilleumier, Nicolas
title: Diagnostic accuracy of Augurix COVID‐19 IgG serology rapid test
date: 2020-07-21
journal: Eur J Clin Invest
DOI: 10.1111/eci.13357
sha: de39e37e07803e02fc2509e95736d6586daa8f47
doc_id: 1030452
cord_uid: c8lv1ze4

AIMS: To validate the diagnostic accuracy of the Augurix SARS‐CoV‐2 IgM/IgG rapid immunoassay diagnostic test (RDT) for COVID‐19. METHODS: In this unmatched 1:1 case‐control study, blood samples from 46 real‐time RT‐PCR‐confirmed SARS‐CoV‐2 hospitalized cases and 45 healthy donors (negative controls) were studied. Diagnostic accuracy of the IgG RDT was assessed against both an in‐house recombinant spike‐expressing immunofluorescence assay (rIFA), as an established reference method (primary endpoint), and the Euroimmun SARS‐CoV2 IgG enzyme‐linked immunosorbent assays (ELISA) (secondary endpoint). RESULTS: COVID‐19 patients were more likely to be male (61% versus 20%; p=0.0001) and older (median 66 versus 47 years‐old; p<0.001) than controls. Whole blood IgG‐RDT results showed 86% and 93% overall Kendall concordance with rIFA and IgG ELISA, respectively. IgG RDT performances were similar between plasma and whole blood. Overall, RDT sensitivity was 88% (95% confidence interval [95%CI]: 70‐96), specificity 98% (95%CI: 90‐100), PPV 97% (95%CI: 80‐100) and NPV 94% (95%CI: 84‐98). The IgG‐RDT carried out from 0 to 6 days, 7 to 14 days and >14 days after the SARS‐CoV‐2 RT‐PCR test displayed 30%, 73% and 100% positivity rates in the COVID‐19 group, respectively. When considering samples taken >14 days after RT‐PCR diagnosis, NPV was 100% (95%CI:90‐100), and PPV was 100% (95%CI:72‐100). CONCLUSIONS: The Augurix IgG‐RDT done in whole blood displays a high diagnostic accuracy for SARS‐CoV‐2 IgG in high COVID‐19 prevalence settings, where its use could be considered in the absence of routine diagnostic serology facilities.

Serology-based assays have not only been shown to be clinically relevant to COVID-19 diagnosis in specific cases, but can also indicate the true nature and extent of the population's exposure to SARS-CoV-2. Although SARS-CoV-2 serological response should not currently be considered a surrogate of individual protection 1 , assessing broad-scale individual COVID-19 serology is likely to be a cornerstone of public health policies during the epidemic after the initial infection-wave.

In this context, the availability of simple, robust and reliable rapid diagnostic tests (RDT) for determining SARS-CoV-2 exposure will be an asset in the diagnostic arsenal for identification of previous SARS-CoV-2 infections. Although tens of RDTs have been developed, only a few validation studies addressing the performance of these tests compared to reference methods have been published so far [2] [3] [4] . We identified three studies of interest. The first showed that the IgG results displayed a 95% specificity in a cohort of PCR-confirmed COVID-19 cases, without comparing the IgG serology results to results obtained with an IgG serology reference method 2 .

Another study had a prospective cohort of 150 patients displaying fever and/or respiratory symptoms and a RDT measuring combined IgM/IgG serology, which had a 71.1% sensitivity and 96.2% specificity against PCR-confirmed COVID-19 diagnosis, resulting in 97.2% PPV and 64.6% NPV 3 . A third study performed in an emergency department on 50 consecutive PCRconfirmed COVID-19 patients admitted for respiratory symptoms during the acute phase of COVID-19, didn't make a strong case for a RDT serology-based diagnosis in these settings, with a sensitivity of 18.4%, a specificity of 91.7%, a NPV of 26.2%, and a PPV of 87.5% 4 .

These studies showed suboptimal performance that can be explained by the accumulation of different biological and analytical factors. At the biological level the timing and the kinetics of the antibody response are key factors, and full seroconversion is likely to take 2-3 weeks. Among analytical factors, the substantial heterogeneity of antigens used in the numerous available immunoassays (including RDTs) is of importance. Indeed these assays rely either on the antibody response against the full Spike protein (S), or against its subdomains 1 (S1) or 2 (S2), against the nucleocapsid (N) or against a combination thereof 5 . If and how the performance of these SARS-CoV-2 assays can be influenced by the antigens used is still unclear, but it can be expected that assays using the combination of full S and N antigens would be optimal at least in terms of sensitivity and may improve specificity. In this respect, a CE/IVD approved RDT using full S and

This article is protected by copyright. All rights reserved N antigen (Augurix; GaDia, Switzerland) has recently been released, but a thorough evaluation of its performances is still lacking.

We aimed to validate this SARS-CoV-2 IgM/IgG serological RDT in a high COVID-19 prevalence setting, using a recently established recombinant immunofluorescence assay as the reference method 6 derived from the reference method for MERS-CoV diagnosis 7 .

In accordance with our institution's ethical committee and national regulations, anonymised leftovers of whole blood-EDTA (hereafter WB-EDTA) were used for this method evaluation.

Only laboratory-based information was used in this study. The reporting of this study conforms to broad EQUATOR guidelines as published by Simera and colleagues 8 . We included blood samples of 46 real-time RT-PCR confirmed COVID-19 cases hospitalized at the University Hospitals of Geneva. These patients had COVID-19 illness severities ranging from moderate to critical, since patients suffering from mild disease were not admitted in this hospital only for the purpose of isolation. We also included 45 unmatched control blood samples from asymptomatic donors without known exposure to SARS-CoV-2, who were not tested by RT-PCR, since they did not meet the testing criteria of our institution. These healthy donors met the blood donation criteria at our institution: age 18 to 65 years old, absence of known acute or chronic infection and without history of cancer, diabetes, or hematological disorders, as well as cardiovascular, autoimmune, inflammatory, chronic kidney or neurological disease. Leftovers from blood specimens (whole blood and plasma) from single patients or controls, collected at a single time point, were tested for various immunoassays. The control blood samples were obtained during the same period (April 2020). WB-EDTA samples were used as a proxy of capillary blood, and were centrifuged (3000g 10 minutes) in parallel to generate EDTA plasma. All analyses (see below) were performed within 72h of blood sampling without any freezing-thawing cycle. The 46 COVID-19 samples were 

platforms, initially including an in house method using eMAG (bioMérieux, France) and the Charité RT-PCR protocol 9 , or the BD SARS-CoV-2 reagent kit for the BD Max system (Becton, Dickinson and Co, US) or Cobas 6800 SARS-CoV-2 RT-PCR (Roche, Switzerland).

This commercially available SARS-CoV-2 IgM/IgG RDT can be used either with capillary blood, whole blood, plasma or serum. One IgM/IgG rapid-test per sample was used. According to the manufacturers' instructions, 20 µl of whole blood (as a proxy for one capillary blood drop) and 10 µl of plasma were applied in parallel for each samples. IgG and IgM responses were read after 10 minutes following manufacturer's instructions, blinded to the reference method results.

Determination of the IgG antibody response against the complete spike (S) protein (both S1 and S2 domains) of SARS-CoV-2 was performed by recombinant immunofluorescence assay (rIFA) as previously validated for MERS-CoV 7,10 and adapted to SARS-CoV-2 6 . Briefly, Vero B4 cells, transfected with the SARS-CoV-2 Spike protein expression vector pCG1-SCOV2-S (kindly provided by M. Hoffmann and S. Pöhlmann, DPZ, Göttingen, Germany), were seeded on multitest glass slides, washed and fixed. After appropriate rehydration and blocking, 30 μl of diluted plasma samples were applied on each spot and incubated for 60 minutes at room temperature (RT). Following washing, secondary goat anti-human-IgG antibody conjugated with Alexa488 (Jackson ImmunoResearch #109-545-088) was applied to each spot and incubated at RT for 45 minutes 6, 10 . The rIFA were results were assessed by two independent readers, blinded to the COVID-19 status, with good inter-observer kappa coefficient 6 .

Euroimmun IgG ELISA uses the S1-domain of the spike protein of SARS-CoV-2 as antigen.

EDTA-plasma was diluted at 1:101 and assessed with the IgG CE-marked ELISA (Euroimmun AG, Lübeck, Germany # EI 2606-9601 G) according to the manufacturer's instructions. The assays were run on Dynex Agility (Ruwag, Switzerland) according to the manufacturer's protocol.

After adding the conjugate, samples' immunoreactivity was measured at an optical density of

This article is protected by copyright. All rights reserved 450nm (OD450) and then adjusted to an internal calibrator to minimize inter-assay variation 6, 11 .

The quantitative results (ratios) were then expressed in arbitrary units and interpreted following the cut-offs derived from our validation study: OD ratio: <0.5 = negative, ≥0.5 and <1.5 = indeterminate, ≥1.5 = positive 6 .

The primary endpoint was to assess the accuracy of IgG detection in whole blood (as a surrogate for capillary blood) by this IgM/IgG RDT against the rIFA reference method (which identifies IgG targeting both S1 and S2 subunits of SARS-CoV-2 Spike protein), within a cohort of 46 RT-PCR confirmed COVID-19 and 45 control blood samples.

The secondary endpoints were to assess: i) the IgG rapid test performances against a commercially available ELISA-based IgG serological immunoassay (Euroimmun), ii) the concordance of the rapid IgM/IgG test results in whole blood versus EDTA-plasma, iii) the performance of the RDT against rIFA and IgG ELISA within each of the COVID-19 DPD subgroups, 0-6, 7-14 and >14

days. Due to the absence of an available comparison method for IgM (not provided by Euroimmun) testing in our institution, we did not formally assess IgM detection performance by the rapid test and its potential added value in individuals infected with SARS-CoV-2.

Vassarstats online tool (www.vassarstats.net) was used to calculate proportions, 95% confidence intervals, median, and Interquartile range (IQR); while significance (p-values) was calculated using Fisher's exact test for categorical variables and the Mann-Whitney U-test for continuous variables. Concordance between immunoassays was assessed with Kendall's coefficient, which was calculated using Statistica (version13.5.0.17, TIBCO Software Inc., Palo Alto, CA, USA).

Vassarstats online tool was also used to determine sensitivity (SN), specificity (SP), and positive and negative predictive values (PPV, NPV). As previously published, indeterminate rIFA IgG and ELISA IgG results were considered to be negative for test performance and concordance analyses, in order to maximize specificity 6 . Statistical significance was defined as p < 0.05.

This article is protected by copyright. All rights reserved

The baseline demographic characteristics of participants were as follows: the 46 RT-PCR confirmed COVID-19 hospitalized patients were older (median 66 years old, IQR 50.5-76) compared to the healthy blood donors (median 47 years old, IQR 39.5-55; p=0.0001). The COVID-19 cohort had a higher proportion of males (n=28, 60.9%) compared to the healthy controls (n=9 males 20%; p= 0.0001). Among COVID-19 patients the median delay between a SARS-CoV-2 RT-PCR diagnostic test and serology testing was 10 days (IQR 5-15 days). The proportion of patients within each DPD subgroup (delta between their molecular testing and serological testing), was 43.5% for 0-6 days (n=20), 30.4% for 7-14 days (n=14) and 26.1% for >14 days (n=12). In the majority of cases (45/46) the RT-PCR tests were done in nasopharyngeal secretions. On one occasion the RT-PCR was carried out on a bronchial aspirate. The median CT value was 23.9 (IQR 19.45-27.55).

Overall, 67% of RT-PCR confirmed COVID-19 patients were IgG seropositive by rIFA, as shown in Table 1 . All patients within the DPD >14 days subgroup were seropositive by rIFA. As expected, the positivity rate of each tested antibody detection method increased in later DPD subgroups. Among the 45 negative controls, 44 (98%, 95%CI: 88-100) were negative in rIFA; one sample was positive in rIFA, as well as in the IgG RDT (in both WB and plasma), and in ELISA IgG immunoassays (ratio 2.81).

IgG RDT accuracy against IgG identification by S-rIFA immunoassays is shown in Tables 2 and   3 As shown in Table 1 , most false-negatives (3 out of 4) were tested in the DPD 0-6 days' subgroup, and none were observed after >14 days DPD. As a secondary endpoint, we analysed the IgG RDT

This article is protected by copyright. All rights reserved performance in each DPD subgroup and in the negative controls. As expected, the longer the delay between a positive PCR and serology testing, the higher the IgG rIFA positivity rate was, as were the IgG RDT SE and NPV, resulting in 100% (both) in the DPD>14 subgroup (Table 3 ). From 7 days post RT-PCR and beyond, the IgG RDT performances against IgG rIFA resulted in 96% SE (95% CI: 76-100) and 98% NPV (95% CI: 88-100).

IgG RDT (WB-EDTA) and Euroimmun IgG ELISA thus showed a concordance coefficient of 0.93 (Kendall , p<0.0001). One false-negative IgG RDT and 1 false-positive IgG RDT were identified, resulting in a SE of 96% (95%CI: 80-100) and SP of 97% (95%CI: 88-99). This resulted in 93% PPV (95%CI: 76-99) and 98% NPV (95%CI: 90-100). The sub-analysis of each DPD is shown in Table 3 .

Out of 91 paired samples, IgG RDT carried out in WB-EDTA compared to plasma gave similar results in 98% of cases (n=89, 95%CI: 92-99). Plasma IgG RDT against rIFA resulted in 88% SE (95CI%: 70-96) and SP 95% (95%CI: 85-99), thus there was no obvious gain when compared to the performances of WB-EDTA IgG RDT against the same reference method.

IgM positivity rates within this cohort were low at 2% for WB-EDTA and 11% for plasma (Table   1) . We cannot establish here if these differences would indicate a significant sensitivity gain in plasma rather than WB (or capillary blood), due to the absence of a reference method (absence of IgM detection by Euroimmun ELISA). Nevertheless, we suspect that, in contrast to IgG, plasma might be more suitable for IgM detection by this IgM/IgG RDT immunoassay.

The key finding of the present validation study, using an unmatched 1:1 case-control design including 50.5% lab-confirmed COVID-19 cases, is that the diagnostic accuracy of IgG Augurix RDT on whole blood when compared to rIFA on plasma displayed a SE of 88%, a SP of 98%, a PPV of 97% and a NPV of 94%. This good Kendall concordance coefficient 12 retrieved with rIFA (0.86) can be explained by the fact that both assays target the immune response against full-length

This article is protected by copyright. All rights reserved S proteins of SARS-CoV-2 5 . Furthermore, our secondary analyses indicated the diagnostic accuracies of this RDT further increased when analyses were done on samples collected after 6 DPD and reached optimal NPV and PPV (100%) after 14 DPD. Such performances indicate that this RDT could be fit for purpose in clinical settings where a high prevalence of COVID-19

prevails, but we need to be very cautious before promoting such tests at large scale. The performance in low prevalence populations still needs to be determined and larger populations need to be tested in particular using whole blood. These results contrast with prior studies using other RDTs which indicated that the PPV and NPV were lower, albeit with a similar proportion of cases versus controls 3, 4 . The reasons underlying such differences are still unknown but can at least be partly explained by inter-assay analytical differences. In addition, one major difference is that the DPD might vary and be very short for the majority of cases in other studies 4 . Of note, the performance of the assay when carried out in the first week after diagnostic is imperfect and potentially limits its use as a potential diagnostic tool during the acute phase. Finally, here we used whole blood and the test was performed in a laboratory environment; we may expect different results in real-life at the bedside and using capillary blood.

The second notable finding of this study lies in the fact that for IgG serology, whole blood-derived results -a validated proxy of capillary blood for antibody assessment 13 -performed as well as plasma. Although expected, this point still needed to be demonstrated for SARS-CoV-2 serology, because it establishes the proof of principle that capillary blood is an adequate medium for SARS-CoV-2 serology. Providing that adequate analytical quality is met by SARS-CoV-2 RDTs, the use of capillary blood alleviates two major pre-analytical hurdles that usually prevent the use of pointof-care devices by the individuals themselves, such venous blood sampling and the need for sample centrifugation. Such operational optimization confers this RDT with the potential to represent an alternative to routine laboratory-assessed COVID-19 IgG serology. In addition, the test was easy to handle and provided clear results, without indeterminate or invalid results, and without inter-operator variability. Whether the use of capillary blood would also be adequate to assess IgM SARS-CoV-2 serology nevertheless remains to be demonstrated.

There are several limitations to this study. First, we present here the results of a method validation study and not a seroprevalence study. Therefore, the PPV obtained here (based on a 50.5%

This article is protected by copyright. All rights reserved proportion of cases defined as laboratory confirmed SARS-CoV-2 by RT-PCR) will probably be lower in a low prevalence setting. We used the date of the RT-PCR diagnosis to perform the DPD subanalysis, and this should not be misinterpreted as days after symptoms onset. Although it is likely an additional limitation of this study, it did provide an objective date to compare the serologies, when symptoms exact onset date can sometimes be difficult to establish. Nevertheless, our results also showed that when targeting a population after one week of diagnosis by RT-PCR, PPV and NPV remained high (above 90%). One control blood sample turned out to be IgG positive by RDT, but also by rIFA and ELISA. Since the results were concordant across the immunoassays tested here, it suggests a possible SARS-CoV-2 serological scar within the control population of blood donors collected during the epidemic period. Although not fully established here, we observed that the RDT results were consistent with the other tests. Another limitation of this validation study lies in its limited sample size leading to broad 95% confidence intervals, requiring confirmation of these data at a larger scale. Finally, our present conclusions only apply to the Augurix RDT, and must not be applied to any other RDTs currently available.

In conclusion, this RDT is not meant to replace a SARS-CoV-2 RT-PCR diagnostic test in the first week of the disease, but could be a reliable option in the toolbox for assessing the SARS-CoV-2 serology in moderate to high COVID-19 prevalence settings, especially in situations where ELISA are not available, or cannot be reliably used. Further investigations in low prevalence situations and using capillary blood are necessary, as are head-to-head comparisons among available RDTs.

The authors have no conflict of interest disclosure to report. Augurix RDTs were provided by Mr P. Ducret (GaDia, Switzerland). GaDia had no role in the study design and realization nor in results interpretation. We thank Erik Boehm for help with editing the manuscript. We are also thankful to Romain Burquier, Krystel Tommasini, Karine Derrac, Lydia Massa-Rouffineau, Myriam Florey-Thévenoz, Marjan Mohitzadeh and Liliane Marchand for technical assistance and help with the serological assays. Last, we would like to thank patients for providing samples and participating in this study. This work was supported by the Division of Laboratory Medicine, HUG and the Center for Emerging Viral Diseases.

Immunity passports" in the context of COVID-19

Evaluation of a COVID-19 IgM and IgG rapid test

an efficient tool for assessment of past exposure to SARS-CoV-2

Clinical evaluation of a rapid colloidal gold immunochromatography assay for SARS-Cov-2 IgM/IgG

Performance of VivaDiag COVID-19 IgM/IgG Rapid Test is inadequate for diagnosis of COVID-19 in acute patients referring to emergency room department

Interpreting Diagnostic Tests for SARS-CoV-2

Validation of a commercially available SARS-CoV-2

Presence of Middle East respiratory syndrome coronavirus antibodies in Saudi Arabia: a nationwide, cross-sectional, serological study

A catalogue of reporting guidelines for health research

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

Accepted Article This article is protected by copyright. All rights reserved 10

Assays for laboratory confirmation of novel human coronavirus (hCoV-EMC) infections

Severe Acute Respiratory Syndrome Coronavirus 2

Specific Antibody Responses in Coronavirus Disease

The measurement of observer agreement for categorical data

Capillary whole blood measurement of islet autoantibodies

 DOA, NV, LK, designed the study, PC, BM, GT, SY, IAV, PLR, LF performed the research, AC,   LM, TA, JLR, JS, IG, SS collected the data, DOA, CAS, IE, LK, NV analyzed the data, and DOA, TA, LK, NV wrote the paper. (100) 12 (100) 12 (100) 12 (100) 12.08 (10.60-13.55) 1. Cut-offs < 0.5 = negative, ≥ 0.5 and < 1.5 = indeterminate, ≥ 1.5 = positive DPD, days post diagnosis; RDT, Augurix rapid diagnostic test; WB, whole blood; rIFA, recombinant immunofluorescence assays