key: cord-0754697-rcee7a4n
authors: Johnson, Marina; Wagstaffe, Helen R.; Gilmour, Kimberly C.; Mai, Annabelle Lea; Lewis, Joanna; Hunt, Adam; Sirr, Jake; Bengt, Christopher; Grandjean, Louis; Goldblatt, David
title: Evaluation of a novel multiplexed assay for determining IgG levels and functional activity to SARS-CoV-2
date: 2020-07-21
journal: bioRxiv
DOI: 10.1101/2020.07.20.213249
sha: 04ea036581ec9365d55186b6efe3f1c4a3b64cf0
doc_id: 754697
cord_uid: rcee7a4n

Background The emergence of SARS-CoV-2 has led to the development of new serological assays that could aid in diagnosis and evaluation of seroprevalence to inform an understanding of the burden of COVID-19 disease. Many available tests lack rigorous evaluation and therefore results may be misleading. Objectives The aim of this study was to assess the performance of a novel multiplexed immunoassay for the simultaneous detection of antibodies against SARS-CoV-2 trimeric spike (S), spike receptor binding domain (RBD), spike N terminal domain and nucleocapsid antigen and a novel pseudo-neutralisation assay. Methods A multiplexed solid-phase chemiluminescence assay (Meso Scale Discovery) was evaluated for the simultaneous detection of IgG binding to four SARS-CoV-2 antigens and the quantification of antibody-induced ACE-2 binding inhibition (pseudo-neutralisation assay). Sensitivity was evaluated with a total of 196 COVID-19 serum samples (169 confirmed PCR positive and 27 anti-nucleocapsid IgG positive) from individuals with mild symptomatic or asymptomatic disease. Specificity was evaluated with 194 control serum samples collected from adults prior to December 2019. Results The specificity and sensitivity of the binding IgG assay was highest for S protein with a specificity of 97.4% and sensitivity of 96.2% for samples taken 14 days and 97.9% for samples taken 21 days following the onset of symptoms. IgG concentration to S and RBD correlated strongly with percentage inhibition measured by the pseudo-neutralisation assay. Conclusion Excellent sensitivity for IgG detection was obtained over 14 days since onset of symptoms for three SARS-CoV-2 antigens (S, RBD and N) in this multiplexed assay which can also measure antibody functionality.

The specificity and sensitivity of the binding IgG assay was highest for S protein with 48 9 Internal quality controls and reference standard reagents were developed from pooled 154 human serum. To measure IgG antibodies, plates were blocked with MSD Blocker A 155 for between 30 minutes and 2 hours then washed three times prior to the addition of 156 reference standard, controls and samples diluted 1:500 in diluent buffer. After 157 incubation for 2 hours with shaking at 700rpm, the plates were washed three times 158 and detection antibody was added at 2 µg/mL (MSD SULFO-TAG™ Anti-Human IgG 159 Antibody). Plates were incubated for 1 hour with shaking and washed three times. 160 MSD GOLD™ Read Buffer B was added and the plates were read using a MESO® 161 SECTOR S 600 Reader. 162

Plates were blocked and washed as above, assay calibrator (COVID-19 neutralising 164 antibody; monoclonal antibody against S protein; 200µg/ml), control sera and test sera 165 samples diluted 1 in 10 in assay diluent were added to the plates. Plates were 166 incubated for 1 hour with shaking at 700rpm. A 0.25µg/ml solution of MSD SULFO-167 TAG™ conjugated ACE-2 was added to unwashed plates followed by incubation for 1 168 hour with shaking, plates were washed and read as above. Percentage inhibition was 169 calculated relative to the assay calibrator; the maximum inhibition reached with 170 calibrator was set as 100% inhibition, minimum at 0.01%. 171 172

Statistical analysis was performed using MSD Discovery Workbench and GraphPad 174

Prism version 8.0 (GraphPad, San Diego, CA). Antibody concentration in arbitrary 175 units (AU) was interpolated from the ECL signal of the internal standard sample using 176 a 4-parameter logistic curve fit. ROC curves showing the sensitivity and specificity 177 (plotted as 100%-specificity %) calculated using each value in the data as a cut-off 178 were plotted for each antigen. A cut-off antibody concentration was chosen based on 179 the lowest value leading to a positive likelihood ratio (LR) of >10, in order to maximise 180 sensitivity while providing strong evidence to rule-in infection (9). For S antigen 181 binding, all LR's were above 10, therefore the LLOD was used as the cut-off for this 182 antigen. Positive predictive value (PPV) was calculated as 183 Comparisons between groups were performed by Kruskal-Wallis one-way ANOVA 187 with Dunn's correction for multiple comparisons. Correlation analysis was performed 188 using Spearman correlation. P values of <0.05 were considered as significant. Latent 189 class models with two classes were fitted with the binary antibody responses as 190 outcome variables, using the poLCA package in the R statistical environment. Figure S1 ). Binding of pooled 212 standard serum to NTD produced low ECL signals and no endpoint titre corresponding 213 to NTD antigen was available for standard serum assignment. The interim values 214 assigned were S 2154 AU, RBD 1837 AU and N 3549 AU. NTD and the remaining 215 antigens were assigned a value of 1000 AU. The focus of this study was the evaluation 216 of the four SARS-CoV-2 antigens only. 217 218 Evaluation of the coronavirus panel for COVID-19 serology 219

The lower limit of detection (LLOD) was assigned as 1% of the standard value in AU, 220 for statistical purposes, values below LLOD were reported as half LLOD (Table 1) . 221

The upper limit of detection (ULOD) was assigned for NTD and RBD only as the S and 222 N antigen did not reach an upper limit (Table 1) . For statistical purposes, ULOD was 223 assigned the highest calculated concentration plus 20%. 224

The coefficient of variation (CV) between duplicates was assessed by analysing 390 225 samples run on 11 plates on 3 different days. All antigens produced a mean CV of 226 <15%, with only NTD falling above the accepted CV of 15% at 17.4% (data not shown). 227

Intra-assay (within plate) and inter-assay (between plate) variation of the assay was 228 assessed by running four samples of varying antibody levels in four replicates on the 229 same plate and across 4 different runs on different days (Supplementary Table 1 ). The 230 mean intra-assay CV was 6.2% and inter-assay variation <15% across all SARS-CoV-231 2 antigens except NTD (19.0%) on one of four samples. 232

To control day to day performance of the assay, a QC sample was run on each plate 233 and an acceptable performance range was set as within 3 SD of the mean. This was 234 determined by running the sample on 8 different plates on 8 different days (average 235 CV 10.3%) ( Table 1) . Table 1 shows the cut-off values selected using our rule 244 of choosing the lowest cut-off with LR>10. For S all LRs were above 10, therefore the 245 LLOD was used as the cut-off for this antigen. NTD data was less consistent than the 246 other SARS-CoV-2 antigens and demonstrated lower sensitivity and specificity ( Figure  247 2D), so this antigen was not evaluated further. 248

The specificity for S, RBD and N assays calculated from the control sera were 97.4% 249 (95%CI 94.1 to 98.9), 92.3% (95%CI 87.6 to 95.3) and 92.8% (95%CI 88.2 to 95.7) 250 respectively (Table 2) . Assay sensitivity was initially calculated on the entire COVID-251 19 cohort; S antigen had the highest AUC and was the most sensitive and specific at 252 90.8% and 97.4% respectively. 253

Using the calculated specificity and sensitivity, the positive and negative predictive 254 values (PPV and NPV) for each antigen at a range of prevalence estimates between 255 0.01 and 0.5 were calculated (Supplementary Figure 3A -B). The PPV and NPV were 256 best for S antigen; for an overall prevalence of 10% the assay has a PPV of 80.4% 257 and NPV of 99.6% for samples taken over 14 days since onset of symptoms, this 258 increased to 92.5% and 98.7% for an overall prevalence of 25%. The assay cut-off determined above was applied and sensitivity and specificity were 270 calculated for groups 0-7 days, over 7 days, over 14 days and over 21 since the onset 271 of symptom for ( Table 2 ). The S antigen was the most sensitive of the three, with a 272 sensitivity of 96.2% and 97.9% >14 days and >21 days since onset of symptoms 273 respectively. 274

The concentration of anti-S, RBD and N antibody all correlated significantly with each 277 other (p<0.0001; Figure ROCs were plotted to visualise the trade-off between sensitivity and specificity for S 296 and RBD neutralisation. Cut-offs (LR>10) were 0.162% for S and 0.524% for RBD 297 (shown by the dotted line on Figure 5A -B). Sensitivity and specificity for S were 97.8% 298 and 97.9% respectively but lower for RBD (77.2% and 92.8% respectively). In the 299 COVID-19 cohort there were some IgG positive sera that did not demonstrate 300 neutralisation (below cut-off, n= 4 for S and 36 for RBD). These sera were 301 predominantly those taken soon after the onset of symptoms; 22 between 0-7 days, 9 302 over 14 days and 5 over 21 days. 303 16 Discussion 304

Accurate tests of SARS-CoV-2 antibodies are critical for reliably evaluating exposure 305 to the virus causing COVID-19. Despite a large number of assays rapidly becoming 306 available, many have not undergone rigorous evaluation. In this study we describe a 307 novel assay that can measure antibody to several SARS-CoV-2 antigens 308 simultaneously as well as evaluating the functional capacity of anti-Spike antibodies. 309

The assay we used is based on existing technology developed by Meso Scale 310

Discovery that uses high binding carbon electrodes in the bottom of 96-well 311 microplates. Each well contains up to 10 antigens bound in discrete spots and bound 312 serum-derived IgG is detected by electro-chemiluminescent labelled (SULFO-TAG) 313

anti-human IgG. Electricity is applied to the plate electrodes leading to light emission 314 by the SULFO-TAG labelled detection antibody and light intensity is measured to 315 quantify analytes in the sample. We decided to evaluate IgG only as the kinetics of 316

IgM responses appear to mimic those of IgG and thus add little value (4). 317

Unlike the majority of studies published to date, we were able to utilise a panel of 318 COVID-19 convalescent plasma recently distributed by WHO to calibrate an internal 319 standard made from pooled convalescent serum. This allowed us to express titres in 320 arbitrary units that can then be compared to other assays that report values calibrated 321 against the WHO panel. The assays performed reliably and consistently over the 322 period of study and passed all the performance criteria expected for a solid-phase 323 based assay with acceptably low inter-and intra-assay coefficients of variation. A QC 324 range established for a medium titre serum gave consistent results throughout the 325 study indicating the stability and repeatability of the platform. 326

Using a carefully defined cohort of known SARS-CoV-2 exposed individuals and 327 relevant controls we were able to show the sensitivity and specificity of the assay for 328 the four antigens of interest. While all antigens had good specificity, the full-length 329 trimeric spike protein had the highest sensitivity, particularly for serum taken more than 330 14 days following the onset of symptoms. Comparing our data for the S and RBD 331 antigens to data in a recently published systematic review and metanalysis of the 332 diagnostic accuracy of serological tests for COVID-19 (10) the trimeric spike assay we 333 evaluated had superior sensitivity to all of the assays included in the review while the 334 RBD antigen performance was superior to most. The reason for this could be related 335

to the technical aspects of the assay itself including the integrity of the antigen used 336 and the sensitivity of the detection platform but also the use of a well-defined cohort 337 of individuals with known exposure to SARS-CoV-2. Only one of the four SARS-CoV-338 2 antigens, the N terminal domain of the spike protein, did not perform well in this 339 assay with poor sensitivity due to the overlap in antibody titres between the COVID-340 19 cohort and controls. 341

The ability to simultaneously measure responses to various SARS-CoV-2 antigens 342 could be seen as an advantage in this type of assay although we did not show an 343

advantage of combined analysis of responses to three antigens compared to using S 344 antigen results alone to predict exposure correctly to the virus. The assay format also 345 permitted the measurement of antibody against spike protein derived from SARS-1, 346 MERS and two seasonal coronaviruses, but the results of antibody binding to these 347 antigens could not be assessed in the same way as for the SARS-CoV-2 antigens due 348 to the absence of defined negative and positive serum sets. 349

A further advantage of this assay is the ability to adapt it for measuring antibody 350 induced inhibition of the interaction between the spike antigen and soluble ACE-2 351 receptor, without the use of live virus and category 3 facilities. This is important as it 352 is thought to be the major mechanism by which SARS viruses, including SARS-CoV-353 18 2 attach to host cell surfaces (11, 12) . In the COVID-19 group, there was a good 354 correlation between the concentration of anti-S and anti-RBD IgG and the inhibitory 355 capacity of serum measured in the pseudo-neutralisaton assay, although a few sera 356 bound antigen but did not neutralize ACE-2 binding. Recently, a study of convalescent 357 serum by Sedoux et al. identified that the majority of antibodies against spike that were 358 generated during the first weeks of COVID-19 infection were non-neutralising and 359 target epitopes outside the RBD (13) which may account for our results. Few of the 360 control cohort sera had any pseudo-neutralisation activity suggesting that pre-existing 361

IgG directed against seasonal Coronavirus spike proteins are unlikely to modify 362 interaction with SARS-CoV-2 although other cross reactive immunological 363

response following exposure to SARS-CoV-2 (14). This pseudo-neutralisation assay 365 has been shown to correlate well with neutralisation assays using live SARS-CoV-2 366 (MSD, personal communication). While plaque reduction neutralisation assays are 367 currently standard for determining host antibody induced viral inhibition, they must be 368 performed in a biosafety level 3 laboratory which limits their widespread use. 369

In summary, the MSD multiplexed coronavirus panel assay evaluated in this study is Acknowledgements 377

The study team would like to thank Meso Scale Discovery for the donation of the plates 378 and reagents that allowed us to complete the work, the COSTARS study team at 379 Table S1 : Intra and inter-assay variability. Within plate (intra) and between plate (inter) 508 Table 3 . 

World Health Organisation

Situation Report -51

Relationship between Anti-Spike Protein Antibody Titers and SARS-CoV-2 In 397

Vitro Virus Neutralization in Convalescent Plasma. bioRxiv

Structural Proteins in Severe Acute 400

Respiratory Syndrome Coronavirus-2

Temporal profiles of viral load in posterior oropharyngeal 406 saliva samples and serum antibody responses during infection by SARS-CoV-407 2: an observational cohort study

Serodiagnostics for Severe Acute Respiratory Syndrome-411

Related Coronavirus-2: A Narrative Review

Prevalence of SARS-CoV-2 in Spain (ENE-COVID): a nationwide, population-418 based seroepidemiological study

A SARS-CoV-2 serological 422 assay to determine the presence of blocking antibodies that compete for 423 human ACE2 binding

Comparison of four new commercial serologic assays for determination of 426 SARS-CoV-2 IgG

Diagnostic tests 4: likelihood ratios

Diagnostic accuracy of serological tests for covid-19: 432 systematic review and meta-analysis

Angiotensin-converting enzyme 2: a 434 functional receptor for SARS coronavirus

Structural and Functional Basis of 437

SARS-CoV-2 Entry by Using Human ACE2

Standard CoV-2 NTD

Graph shows ECL signal obtained from a serial dilution series (1 in 100, then 1 in 4 serial dilution) of standard serum and NIBSC control sera 20/130 and 10/124. NIBSC control serum 20/130 was used to assign values to standard serum for SARS-CoV-2 spike (S) and receptor binding domain (RBD) and NIBSC control serum 20/124 was used to assign a value to SARS-CoV-2 nucleocapsid (N). No endpoint titre corresponding to NTD antigen was available for standard serum assignment.