key: cord-0756996-rjxraw56
authors: Ekström, N.; Virta, C.; Haveri, A.; Dub, T.; Hagberg, L.; Solastie, A.; Österlund, P.; Vihervaara, T.; Erlund, I.; Nohynek, H.; Melin, M.
title: Analytical and clinical evaluation of antibody tests for SARS-CoV-2 serosurveillance studies used in Finland in 2020
date: 2021-01-25
journal: nan
DOI: 10.1101/2021.01.21.21250207
sha: f33af853a74e22ae5897073d72bf5b4d33f3858c
doc_id: 756996
cord_uid: rjxraw56

Background: Sensitive and highly specific antibody tests are critical for detection of SARS-CoV-2 antibodies especially in populations where seroprevalence is low. Aim: To set up, optimize and evaluate the analytical and clinical performance of a new in-house microsphere immunoassay for measurement of IgG antibodies to SARS-CoV-2 nucleoprotein for assessment of population seroprevalence in Finland. Methods: We set up a new in-house microsphere immunoassay (FMIA) with SARS-CoV-2 nucleoprotein and optimized its analytical performance. For evaluation of clinical performance, we tested sera collected in a well-characterized cohort of PCR positive-confirmed SARS-CoV-2 patients (n=89) with mostly mild symptoms, and before the COVID-19 pandemic (n=402), for nucleoprotein specific IgG concentrations by FMIA and a commercial chemiluminescent immunoassay and for neutralizing antibodies by the microneutralization test. Results: The analytical performance of FMIA was established in terms of sensitivity, linearity and precision. FMIA discriminated between COVID-19 patient and control samples with high specificity (100%) and sensitivity (100%). We generated FMIA seropositivity cut-offs, 0.46 and 1.71 U/ml, for low- and high-seroprevalence settings, respectively. In addition, we obtained high level of agreement between FMIA results and results by the microneutralization test. Conclusion: The fluorescent microsphere immunoassay showed excellent analytical and clinical performance and is well suited for serosurveillance studies of SARS-CoV-2. However, to optimize analytical sensitivity and clinical specificity of the assay, different seropositivity thresholds depending on the intended use of the assay and the target population, may be needed.

The newly emerged SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) has resulted in worldwide pandemic. Detection of antibodies against SARS-CoV-2 is essential in population level surveillance studies and in outbreak monitoring as antibodies reveal evidence of previous infection. A test that measures the presence of neutralizing antibodies against SARS-CoV-2 is considered the golden standard. Several studies have demonstrated that SARS-CoV-2 infection is able to induce neutralizing antibodies in most subjects and that the strength of the immune response correlates with disease severity [1] [2] [3] [4] . However, it is possible that COVID-19 patients who experience mild symptoms may not have detectable levels of neutralizing antibodies [5] . The development of humoral immunity in COVID-19 patients seems to take a median of two weeks after onset of symptoms [6] [7] [8] . While neutralizing antibodies mainly target the S protein, antibodies to nucleoprotein could serve as a sensitive measure of past infection, because nucleoprotein is produced most abundantly during the viral replication. Most vaccines and vaccine candidates target the S protein. An antibody test based on nucleoprotein would enable differentiation between immunity induced by infection and vaccination and could thus be used to monitor the prevalence of infections also in vaccinated populations.

When the number of infections in the population is low, even a relatively accurate antibody test will yield a significant number of false positive results, which could be caused by cross-reactive antibodies induced by endemic coronaviruses or other infections. The sensitivity and specificity of the antibody tests should be carefully evaluated and taken into account not to overestimate the population seroprevalence [9] .

In this study we describe evaluation of analytical and clinical performance of an in-house microsphere immunoassay (FMIA) for assessment of population seroprevalence to SARS-CoV-2 in Finland. We measured serum IgG antibodies to nucleoprotein in a well-characterized cohort of PCR positive-confirmed COVID-19 patients and population samples collected before the COVID-19 pandemic. Furthermore, we measured neutralizing antibodies by microneutralization test using SARS-CoV-2 virus cultures. We focused on serum samples collected during the late convalescent phase, because appearance of IgG antibodies in serum takes on average two weeks following SARS-CoV-2 infection. In addition, we analysed the samples by a widely used commercial automated chemiluminescence immunoassay and compared the clinical performance of the assays. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; https://doi.org/10.1101/2021.01.21.21250207 doi: medRxiv preprint

SARS-CoV-2 infection, exposure in households and production of antibodies to SARS-CoV-2 within families was investigated in Helsinki, Finland. Altogether 129 subjects from 39 households participated and gave written consent between March and June 2020. Serum samples (n=312) collected during the study were used for assessment of analytical performance of the antibody assays. For assessment of clinical performance, serum samples (n=89) from 58 subjects with PCR positive-confirmed SARS-CoV-2 infection collected during the late convalescent phase at a median of 30 days from the onset of symptoms (range 15 to 51 days) were used. The majority of the COVID-19 patients reported mild symptoms. Four patients reported severe symptoms and required hospital treatment. As a control group, serum samples collected in 2019 before the COVID-19 pandemic from 402 Finnish subjects aged 1 to 90 years of age were assessed.

Demographics of the COVID-19 patients and donors for the pre -COVID-19 serum panel (residual sera from Helsinki University Hospital laboratory, HUSLAB) are described in Supplementary Tables S1 and S2 .

Fluorescent microsphere immunoassay (FMIA) An in-house fluorescent microsphere immunoassay (FMIA) modified from a previously described method

[10] was set up and optimized (Supplementary material). In brief, pre-COVID-19 sera diluted 1/100 and sera from COVID-19 patients diluted 1/100 and 1/1600, in-house reference serum (1/400 to 1/1 638 400) and two control sera per plate were mixed with SARS-CoV-2 nucleoprotein conjugated fluorescent microspheres. IgG antibodies were subsequently detected by R-Phycoerythrin (RPE) -conjugated secondary antibody. The plate was read on MAGPIX® system using xPONENT software v4.2 (Luminex®Corporation, Austin, TX). The in-house reference sample was assigned an arbitrary IgG antibody concentration of 100.

Median fluorescent intensity (MFI) was converted to FMIA unit/ml (U/ml) by interpolation from a 5- is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; https://doi.org/10.1101/2021.01.21.21250207 doi: medRxiv preprint parameter logistic (5-PL) reference curve. When serum was analysed in two dilutions, the IgG concentration was calculated for the 1/100 dilution and as the mean concentration of the two dilutions.

Abbott SARS-CoV-2 IgG test (Abbott test)

The quantitative analysis of SARS-CoV-2 IgG antibodies directed against the nucleoprotein was carried out 

The cytopathic effect (CPE)-based microneutralization test (MNT) was performed as previously described [11] . SARS-CoV-2-specific neutralizing antibodies were detected from 2-fold serially diluted serum samples, starting from 1/4, using hCoV-19/Finland/1/2020 (GISAID accession ID EPI_ISL_407079) and hCoV-19/Finland/FIN-25/2020 (EPI_ISL_412971) viruses imported to Finland from China and Italy, respectively.

Results were expressed as MNT titers corresponding to reciprocal of serum dilution that inhibited 50% of SARS-CoV-2 infection observed by CPE of inoculated cells. MNT titer was considered positive when ≥6 by either or both of the SARS-CoV-2 strains and borderline when 4≤ titer<6 by both strains.

A serum panel from COVID-19 patients was used to quantify limits of detection (LOD) and quantification (LOQ), and assess linearity and precision of FMIA. LOD and LOQ (U/ml) were assessed by averaged MFI is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; https://doi.org/10.1101/2021.01.21.21250207 doi: medRxiv preprint values generated from blank wells (n=104) and from this, mean + 3 and 10 standard deviations calculated, respectively, and interpolated from the 5-PL reference curve with GraphPad Prism v7. To assess linearity, FMIA was performed with five 4-fold dilutions (1/100 to 1/1600) of six sera, and the resulting log2-transformed IgG concentrations were compared by linear correlation analysis. Optimal linearity would yield Pearson correlation coefficient R 2 ≥0.95. Additionally, linearity was assessed by comparing IgG concentrations obtained by dilutions 1/100 and 1/1600 of 312 sera.

Precision of FMIA was assessed by intra-assay (repeatability) and inter-assay variation (intermediate precision). Repeatability was determined by assaying 16 sera (range 0.04 to 5.92 FMIA U/ml) in duplicate and by assaying six reference serum dilutions in duplicate within a plate on 62 plates and between two plates analysed on the same day (n=26). The coefficient of variation (CV) between duplicate results was calculated and averaged. Intermediate precision (day to day) was assessed by seven sera in five separate runs, by comparing concentrations of six reference serum dilutions in 26 runs and of two control sera on 54 runs on different days. The CV between results obtained on different days was calculated and averaged.

Intermediate precision was additionally assessed between results obtained by four different batches of conjugated microspheres, by three lots of RPE-conjugated secondary antibody and by four technicians with panels of sera (n=14-39). CV≤ 20% was considered acceptable precision.

The analytical performance of the Abbott test was evaluated by quantifying accuracy and precision. In the absence of a standard reference serum specimen accuracy was evaluated by comparing the replicate values is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; considered acceptable precision.

Analytical performance of MNT was evaluated by assessing precision. Inter-assay variation of MNT was evaluated by calculating CV of the titers of a positive in-house control serum included in each assay.

Intermediate precision of MNT was assessed by analyzing a panel of sera (n=98) at two time points five months apart and calculating mean CV of the titers.

The ability of FMIA to discriminate PCR-positive confirmed COVID-19 patients and controls was assessed for is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; https://doi.org/10.1101/2021.01.21.21250207 doi: medRxiv preprint

The investigations were carried out in accordance with the General Data Protection Regulation (Regulation 

The analytical sensitivity of FMIA was defined by the limit of detection as 0.02 FMIA U/ml and the limit of For assessment of precision, the mean CV for the intra-assay variation within a plate was 7 % and 8% between plates analysed on the same day. Inter-assay variation, determined for IgG concentrations of seven samples run on five days, and for two control samples on 54 days, was 13% and 19%, respectively. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; https://doi.org/10.1101/2021.01.21.21250207 doi: medRxiv preprint of conjugated microspheres ranged from 12 to 20%, for three different lots of RPE-conjugated secondary antibody CV was 10 %, and between results obtained by four technicians CV ranged from 10 to 25 %.

Overall, both intra-and inter-assay variation was found acceptable, and the precision of the FMIA established. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; https://doi.org/10.1101/2021.01.21.21250207 doi: medRxiv preprint

Because linearity analysis indicated that serum dilution 1/100 generates lower IgG concentrations than more diluted serum, the clinical specificity and sensitivity of FMIA were evaluated using two different IgG concentration cut-offs generated by ROC analysis. IgG concentration data using 1/100 diluted sera collected from COVID-19 patients and controls generated ROC curve with an area under curve 0.998 (95% CI 0.995-1). The ROC generated cut-off 0.46 U/ml resulted in 97.5% specificity (Wilson/Brown 95% CI 95.5-98.6).

Sensitivity of FMIA was 96.0% (95% CI 80.5, 99.3), 100% (77.2, 100.0) and 100% (92.9, 100.0) for samples collected 15-21, 22-28 and >28 days after onset of symptoms, respectively. When ROC analysis was performed using IgG concentration data obtained from the mean concentration calculated from 1/100 and ≥1/1600 diluted serum from COVID-19 patients it yielded an area under curve 1.00 (95%CI 1.00, 1.00) and generated cut-off of 1.71 U/ml. This latter cut-off resulted in 100% specificity (95% CI 99.1 -100%) and

100% sensitivity for samples taken 15-21, 22-28 and >28 days after onset of symptoms (Table 1, Figure 1 ).

All sera collected before the COVID-19 pandemic were identified as true negative by the Abbott test resulting in 100 % specificity (Table 2, Figure 1 ). To assess sensitivity, the test identified 69/73 samples as is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; https://doi.org/10.1101/2021.01.21.21250207 doi: medRxiv preprint

All sera collected before the COVID-19 pandemic were identified as true negative by the MNT resulting in 100 % specificity (Table 3, Figure 1 ). The sensitivity of MNT was 96.0%, 100% and 100.0% for samples taken 15-21, 22-28 and >28 days post onset of symptoms at sample collection ( Table 3 ). The SARS-CoV-2 strains imported from China and Italy identified 98% and 94% of samples as true positive, respectively, and both strains identified 93% of the samples as true positive. One sample, taken 16 days post onset of symptoms, was identified as false negative by both strains, but had a borderline titer 4 to 6.

All three assays identified each serum taken before the COVID-19 pandemic as true negative resulting in equal specificity of the assays. However, sensitivity of the assays varied by timing of sample collection and between assays. Sensitivity varied most for samples taken <22 days after onset of symptoms. FMIA with cut-off 1.71 U/ml solely identified samples taken <22 days post onset of symptoms as true positive suggesting high analytical sensitivity. The results obtained by the quantitative assays, FMIA and the Abbott test, showed strongest correlation (r=0.67-0.90, P<0.05; Figure 2C ). Furthermore, results obtained by FMIA and the Abbott test correlated modestly, but statistically significantly with MNT titers for samples taken 15 to 21, and >28 days after onset of disease (Figure 2A-B) . The highest agreement between FMIA and MNT was seen for samples taken >28 days after onset of symptoms. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ;

population. Knowledge on the performance of the assays used in seroprevalence studies is essential to provide reliable seroprevalence data. In this study, we described the development of a fluorescent microsphere immunoassay for detection of antibodies against SARS-CoV-2 nucleoprotein, used in evaluation of SARS-CoV-2 seroprevalence in Finland. The analytical performance of FMIA was established in terms of sensitivity, linearity and precision, and it was able to discriminate samples from COVID-19 patients with high specificity and sensitivity. In addition, we showed high level of consistency of the FMIA results to results obtained by the microneutralization test.

The accuracy of reporting specificity and sensitivity of an assay is greatly dependent on clinical sample selection for the evaluation. Previous reports show that milder COVID-19 disease results in lower levels of antibodies and later seroconversion than shown in severe disease [3, 12, 13] . The statistical outcome may not be the same when using sera of from COVID-19 patients with severe symptoms or sera from population-based serosurveillance studies where the majority of subjects will not develop severe symptoms. This increases the importance of high specificity of an assay used in population seroprevalence studies. The majority of sera from COVID-19 patients used for assessment of specificity and sensitivity in our study were from patients with mild symptoms. This is in agreement with the intended use of the test i.e. seroprevalence studies with low-disease prevalence situation.

The FMIA was developed for assessment of seropositivity in population. In the prevailing lowseroprevalence setting it was essential to optimize the analytical sensitivity and clinical specificity of the assay. This could be done using low serum dilution (1/100) and seropositivity cut-off of 0.46 U/ml, but would result in clinical sensitivity of 97.5% for samples collected 15-22 days post onset of symptoms. Thus, to optimize analytical sensitivity and clinical specificity, FMIA could be used as the first screening test and be supplemented with another assay, such as neutralization test or a second test based on spike protein antigens to exclude false positive results. By using the ROC generated higher seropositivity cut-off 1.71 . CC-BY-NC-ND 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; https://doi.org/10.1101/2021.01.21.21250207 doi: medRxiv preprint U/ml, both clinical specificity and sensitivity of FMIA could be optimized to 100%. This FMIA cut-off is ideal when high seropositivity is expected. Consequently, the optimal threshold of FMIA depends on the intended use of the test and the target population. Moreover, we showed that low serum dilutions (1/100, 1/200) resulted in lower antibody concentrations than higher serum dilutions (≥1/1600). This should be taken into account when comparing antibody concentrations over time or between groups; samples should be analysed by the same serum dilutions or the concentration calculated as the mean of more than one serum dilution.

The clinical performance of the Abbott test has been assessed in various previous studies which have reported relatively high specificity, but varying sensitivity with regard to disease severity and time between symptom onset and sampling [14, 15] . Using the manufacturer's recommended index value cutoff for seropositivity we reported an assay specificity of 100% and sensitivity ≥85.7% at ≥15 days post onset of symptoms. For samples taken <22 days after onset of symptoms we reported lower sensitivity than given by the manufacturer.

In this study with majority of patients having mild disease, all three assays had sensitivities comparable to those reported for hospitalized patients (for samples taken 22 to 51 days post symptom onset). However, differences in assay sensitivity may become apparent later on when antibody levels decline. A wide range of SARS-CoV-2-neutralizing antibody titers have been reported after infection and these vary depending on the length of time from infection and the severity of disease [16] . For those who develop low neutralizing antibody response, titers have been reported to return to baseline in three to four months whereas binding antibody levels were maintained [17] . This suggests that over time the clinical sensitivity of MNT may decrease emphasizing the importance of assays measuring binding antibodies. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ;

Limitations of this study include that the sample size, especially for assessment of sensitivity by days post onset of symptoms, was relatively small. Another limitation was that because only few patients had been hospitalized, we could not assess differences in performance that depend on disease severity. On the other hand, as the majority of the samples used in assessment of performance were from COVID-19 patients with mild symptoms, they well reflect the general population in seroprevalence studies. Furthermore, testing potentially cross-reactive sera from patients with recent infection of endemic human coronaviruses or other respiratory pathogens was not done due to lack of suitable sera.

In the future, the capacity of the FMIA-based assays can be utilized in antibody testing within SARS-CoV-2 vaccine evaluation. The flexibility of FMIA will allow several additional SARS-CoV-2 antigens to be incorporated easily into a single multiplex assay. Importantly, by measuring antibodies to the SARS-CoV-2 nucleoprotein, immune responses induced by infection and potential circulation of the virus can be assessed also in vaccinated populations. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; Table 3 . Clinical sensitivity (NPA) and specificity (PPA) of the MN test using titer ≥6 as the cut-off. Sensitivity was determined according to days post onset of symptoms (DPO) at sample collection; 15-21, 22-28 and >28 days. Positive (PPV) and negative (NPV) predictive values at 1% prevalence are shown. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted January 25, 2021. ; https://doi.org/10.1101/2021.01.21.21250207 doi: medRxiv preprint

Convergent antibody responses to SARS-CoV-2 in convalescent individuals

Neutralizing antibody response in mild COVID-19

Clinical and immunological assessment of asymptomatic SARS-CoV-2 infections

Antibody responses to SARS-CoV-2 in patients with COVID-19

Evaluating the association of clinical characteristics with neutralizing antibody levels in patients who have recovered from mild COVID-19 in

Profiling early humoral response to diagnose novel coronavirus disease (COVID-19)

Magnitude and kinetics of anti-SARS-CoV-2 antibody responses and their relationship to disease severity

Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease

Meta-analysis of diagnostic performance of serological tests for SARS-CoV-2 antibodies and public health implications

Serological and molecular findings during SARS-CoV-2 infection: the first case study in Finland

Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Responses in Coronavirus Disease Patients

SARS-CoV-2-specific antibody detection for seroepidemiology: A multiplex analysis approach accounting for accurate seroprevalence

Clinical performance of two SARS-CoV-2 serologic assays

Longitudinal characterization of the IgM and IgG humoral response in symptomatic COVID-19 patients using the Abbott Architect

Longitudinal observation and decline of neutralizing antibody responses in the three months following SARS-CoV-2 infection in humans

Longitudinal analysis of serology and neutralizing antibody levels in COVID19 convalescents

We gratefully acknowledge the authors, originating and submitting laboratories of the sequences from GISAID's database. All submitting laboratories may be contacted directly via the GISAID website www.gisaid.org.

Conflict of interest: NE, CV and MM are co-investigators in an unrelated study, for which THL has received research funding from GlaxoSmithKline Vaccines. The other authors report no potential conflicts of interest.Funding statement: The study was supported by funds from the Finnish Institute for Health and Welfare (THL)