key: cord-0848722-d2tf2g7f authors: Brochot, Etienne; Demey, Baptiste; Handala, Lynda; François, Catherine; Duverlie, Gilles; Castelain, Sandrine title: Comparison of different serological assays for SARS-CoV-2 in real life date: 2020-08-02 journal: J Clin Virol DOI: 10.1016/j.jcv.2020.104569 sha: 6d275c79f523e6db51553c77087e20cf39243d5e doc_id: 848722 cord_uid: d2tf2g7f BACKGROUND: The emergence of the global SARS-CoV-2 pandemic required the rapid and large-scale deployment of PCR and serological tests in different formats. OBJECTIVES: Real-life evaluation of these tests is needed. Using 168 samples from patients hospitalized for COVID-19, non-hospitalized patients but infected with SARS-CoV-2, patients participating in screening campaigns, and samples from patients with a history of other seasonal coronavirus infections, we evaluated the clinical performance of 5 serological assays widely used worldwide (WANTAI®, BIORAD®, EUROIMMUN®, ABBOTT® and LIAISON®). RESULTS: For hospitalized patients, all these assays showed a sensitivity of 100 % from day 9 after the symptoms onset. On the other hand, sensitivity was much lower for patients who did not require hospitalization for COVID-19 confirmed by PCR (from 91.6 % for WANTAI® to 69 % for LIAISON®). These differences do not seem to be due to the antigens chosen by the manufacturers but more to the test formats (IgG detection versus total antibodies). In addition, more than 50 days after a positive PCR for CoV-2-SARS the proportion of positive patients seem to decrease. We did not observe any significant cross-reactions for these techniques with the four other seasonal coronaviruses. CONCLUSION: In conclusion, the evaluation and knowledge of the serological tests used is important and should require an optimized strategy adaptation of the analysis laboratories to best meet patient’s expectations in the face of this health crisis. Since the outbreak of coronavirus cases worldwide, a frantic race for the availability of PCR and serological tests has been launched by the entire community of in vitro diagnostic manufacturers (2) . Antibody tests, such as enzyme-linked immunosorbent assays (ELISA) or chemiluminescent assays (CLIA), can overcome some of these difficulties. Serological tests can detect past infection with CoV-2-SARS in patients for whom PCR could not be performed or for whom the nasopharyngeal swab result was falsely negative (3) . For serological tests, manufacturers have often demonstrated very good performance in terms of sensitivity and specificity (4, 5) . However, for antibody testing in acute disease, the sensitivity is highly dependent on the kinetics of antibody development. Similarly, specificity is dependent on the type of samples selected to evaluate cross-reactions. It is necessary to evaluate these cross-reactions to other viruses of the coronavirus family. In J o u r n a l P r e -p r o o f addition, firms have adopted different strategies in terms of selecting their antigenic base and the type of immunoglobulins detected. The rapid availability of these tests then requires on-site evaluation by users to detect flaws in the results (6, 7) . Thus, we evaluated five commercial serological tests widely used worldwide on samples from patients hospitalized for COVID-19, non-hospitalized patients but infected with SARS-CoV-2, patients participating in screening campaigns, and samples from patients with a history of other seasonal coronavirus infections. J o u r n a l P r e -p r o o f The study was conducted at Amiens University medical Center. The study was approved by the institutional review board of the Amiens University Medical Center (number PI2020_843_0046, 21 April 2020). Samples were derived from de-identified excess serum specimens sent to our clinical virology lab. Patient serum samples used in this study were submitted to the routine serology laboratory. The assays were validated using serum samples from (i) patients hospitalized for COVID-19 (n=20), non-hospitalized patients but PCR confirmed with SARS-CoV-2 (n= 58), patients participating in screening campaigns (n= 62), and samples from patients with a history of other seasonal coronavirus infections (n= 28). The list and characteristics of the different serological tests evaluated are listed in Table 1 . The antigen used in the assay is SARS-CoV-2 nucleocapsid for ABBOTT® and BIORAD®, Spike 1 for EUROIMMUN®, Spike 1 and 2 for LIAISON® and receptor binding domain (RBD) for WANTAI®. ABBOTT®, EUROIMMUN® and LIAISON® detect immunoglobulin G while BIORAD® and WANTAI® detect total antibodies with J o u r n a l P r e -p r o o f double antigen bridging assay (DABA). A sample with a doubtful signal was tested a second time and if the result was still the same, the result was considered negative for our evaluation. The demographic information of the 168 patients (sex, age) was extracted from the patient data software (detailed in supplementary table 1). Sensitivity was defined as the proportion of patients correctly identified as having SARS-CoV-2 infections. Percent of agreement and Kappa index were calculated with GraphPad software v5.1. All samples from the 4 patient groups were run through the 5 CoV-2 SARS antibody detection kits. For the first group, with 20 patients hospitalized for COVID-19 with a positive nasopharyngeal SARS-CoV-2 PCR, all samples were positive with these serological assays evaluated ( Figure 1A ). Then for patients also screened for COVID-19 but not hospitalized and patients participating in screening campaigns, disparities between the tests were found. The figures ranged from 91.6% (WANTAI®) to 69% (LIAISON®) for the first group and from 40.3% (WANTAI®) to 21% (LIAISON®) for the second. These differences do not seem to be due to the antigens chosen by the manufacturers but more to the test formats (IgG detection versus total antibodies). We For these 168 samples divided into 4 groups for which the five serological techniques were performed, we compared the number of positive samples two by two and calculated the overall percent agreement (negative and positive samples) and Kappa index ( Table 2) . For positive samples, for all sera, the highest number was for J o u r n a l P r e -p r o o f WANTAI®/BIORAD® (n= 93) while the lowest number was for LIAISON®/EUROIMMUN® and LIAISON®/ABBOTT® (n= 70) ( Table 2A) . The BIORAD® and ABBOTT® techniques using the Nucleocapsid as an antigenic base had a good percentage of positive approvals (91.4%) and a high kappa index (0.82) (Table 2B and C). All kappa indices were above 0.6 but with a range from 0.61 (LIAISON®/BIORAD®) to 0.82 (BIORAD®/ABBOTT®). Finally, in order to better study the discrepancies in the positive results in the two groups for which we observed differences (Outpatients with SARS-CoV-2 positive PCR and Outpatients with no history of SARS-CoV-2 infection) , we mentioned the indexes on two-by-two comparison histograms (supplementary figure 1). We observe most of the time for the positives samples with one assay, that these index numbers are low and rarely at signal saturation. Finally, we analyzed more finely for these two groups the number of positive tests (from 1 to 5). We obtained for the 88 positive sera by the most sensitive technique (Wantai), 48 (55%) sera positive with the five different assays (Table 3) . However, for the rest of the positive samples, we observed significant differences between assays, and in particular for the LIAISON® assay , 90.5% (48/53) samples found positive with this technique were also positive with the four other assays. In this study, using 168 samples from a diverse group of patients in the SARS-CoV-2 pandemic,, we compared the performance of five widely used serological tests from around the world. Many serological tests in different formats are now available and evaluated by different authorities but never with the same panel of samples. This allows us to compare these tests under real-life conditions with different categories of patients. It is clear that for patients requiring hospitalization for COVID-19, the humoral response to CoV-2-SARS is so exacerbated that all properly developed techniques will have 100% sensitivity. The sensitivity problem can arise under two conditions. The first is when the antibody detection is too early in the course of the infection, especially for techniques that detect only IgG. The second condition concerns a percentage of SARS-CoV-2 infection with asymptomatic or mild forms, in which case IgG synthesis is absent or low while IgM is probably more frequently detected. Moreover, for this sensitivity problem, manufacturers have had to make new devices available in record time but probably with a preference for specificity over sensitivity in order not to suffer from bad publicity in case of false positive reactions. For example, we have tested the EUROIMMUN® IgA kit which shows a specificity of 90% on the package leaflet which we have confirmed (data not shown). Antibody testing may therefore be relevant in the following settings: i) diagnosis of patients who seek medical attention more than a week after the onset of J o u r n a l P r e -p r o o f symptoms; ii) contact tracing; iii) determining potential immunity and risk of infection; and iv) sero-epidemiological studies to understand the extent of COVID-19 spread. There is a debate as to whether sensitivity or specificity should be preferred for an acute disease for which serology can only provide mainly epidemiological data. Perhaps we will soon have more sensitive techniques while maintaining a good specificity. In terms of specificity, which we evaluated against other seasonal coronaviruses, all techniques gave excellent results. The different manufacturers have excellent specificity figures, but these must then be evaluated under real conditions because of the diversity of possible reactions and the non-exhaustive search for potential cross-reactions. As observed in Figure 1B on samples more than 50 days post-PCR, the percentage of positive results tends to decrease as recently described for neutralizing antibodies (8, 9) . All these results raise the question of the role of humoral immunity in relation to cellular immunity in combating this infection and its persistence (10, 11) . Although we cannot compare the periods in this Figure 1B This work was supported by a grant from the Amiens university Medical center. CoV-2 positive 44 35 50 28 F SARS-CoV-2 positive NS 43 51 57 M SARS-CoV-2 positive 68 61 52 41 F SARS-CoV-2 positive 68 60 53 51 F SARS-CoV-2 positive 69 57 54 31 M SARS-CoV-2 A pneumonia outbreak associated with a new coronavirus of probable bat origin Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV-2) The important role of serology for COVID-19 control Comparison of the diagnostic sensitivity of SARS-CoV-2 nucleoprotein and glycoprotein-based antibody tests Performance Characteristics of the Abbott Architect SARS-CoV-2 IgG Assay and Seroprevalence in An evaluation of COVID-19 serological assays informs future diagnostics and exposure assessment Dynamic profile for the detection of anti-SARS-CoV-2 antibodies using four immunochromatographic assays Anti-Spike, anti-Nucleocapsid and neutralizing antibodies in SARS-CoV-2 inpatients and asymptomatic carriers Longitudinal evaluation and decline of antibody responses in SARS-CoV-2 infection Do Not Minimize the Impact of Cellular Immunity in the Development of Vaccines and Therapeutics for COVID-19 Types of Assays for SARS-CoV-2 Testing: A Review