key: cord-1033082-1193k3gc
authors: Heaney, Christopher D.; Pisanic, Nora; Randad, Pranay R.; Kruczynski, Kate; Howard, Tyrone; Zhu, Xianming; Littlefield, Kirsten; Patel, Eshan U.; Shrestha, Ruchee; Laeyendecker, Oliver; Shoham, Shmuel; Sullivan, David; Gebo, Kelly; Hanley, Daniel; Redd, Andrew D.; Quinn, Thomas C.; Casadevall, Arturo; Zenilman, Jonathan M.; Pekosz, Andrew; Bloch, Evan M.; Tobian, Aaron A.R.
title: Comparative performance of multiplex salivary and commercially available serologic assays to detect SARS-CoV-2 IgG and neutralization titers
date: 2021-10-09
journal: J Clin Virol
DOI: 10.1016/j.jcv.2021.104997
sha: 4efcb8234988e7a3dea08bae66a2ab0f7139acee
doc_id: 1033082
cord_uid: 1193k3gc

Oral fluid (hereafter saliva) offers a non-invasive sampling method for detection of SARS-CoV-2 antibodies. However, data comparing performance of salivary tests against commercially-available serologic and neutralizing antibody (nAb) assays are lacking. This study compared the performance of a laboratory-developed multiplex salivary SARS-CoV-2 IgG assay targeting antibodies to nucleocapsid (N), receptor binding domain (RBD) and spike (S) antigens to three commercially-available SARS-CoV-2 serology enzyme immunoassays (EIAs) (Ortho Vitros, Euroimmun, and BioRad) and nAb. Paired saliva and plasma samples were collected from 101 eligible COVID-19 convalescent plasma (CCP) donors >14 days since PCR+ confirmed diagnosis. Concordance was evaluated using positive (PPA) and negative (NPA) percent agreement, and Cohen's kappa coefficient. The range between salivary and plasma EIAs for SARS-CoV-2-specific N was PPA: 54.4-92.1% and NPA: 69.2-91.7%, for RBD was PPA: 89.9-100% and NPA: 50.0-84.6%, and for S was PPA: 50.6-96.6% and NPA: 50.0-100%. Compared to a plasma nAb assay, the multiplex salivary assay PPA ranged from 62.3% (N) and 98.6% (RBD) and NPA ranged from 18.8% (RBD) to 96.9% (S). Combinations of N, RBD, and S and a summary algorithmic index of all three (N/RBD/S) in saliva produced ranges of PPA: 87.6-98.9% and NPA: 50-91.7% with the three EIAs and ranges of PPA: 88.4-98.6% and NPA: 21.9-34.4% with the nAb assay. A multiplex salivary SARS-CoV-2 IgG assay demonstrated variable, but comparable performance to three commercially-available plasma EIAs and a nAb assay, and may be a viable alternative to assist in monitoring population-based seroprevalence and vaccine antibody response.

As coronavirus disease 2019 (COVID-19) emerged, there were limited diagnostic and treatment options. Antibody titers can be determined by testing blood using commercially available enzyme immunoassays (EIAs) that typically measure antibody responses to a single antigen. Alternatively, microneutralization assays can be employed to determine a neutralizing antibody (nAb) titer. However, microneutralization requires both intensive biosecurity measures and substantial time, which are not amenable to high throughput monitoring of SARS-CoV-2 antibody responses at population scale over time.

Antibodies to SARS-CoV-2 have been evaluated in oral fluid (hereafter saliva), but little is known about how antibody titers in saliva correlate with those measured using plasma serologic assays for detection of SARS-CoV-2-specific IgG and nAb activity [1] [2] [3] . If comparable performance, saliva would offer several advantages over blood-based testing: collection is non-invasive and can be self-administered. These advantages would improve the scale and efficiency of screening, population-based surveillance and assessment of vaccine responsiveness. This study sought to evaluate the performance of a multiplex salivary SARS-CoV-2 IgG assay relative to three commercially-available EIAs, and a nAb assay.

This study used stored samples and data from two parent studies that were approved by The Johns Hopkins University School of Medicine Institutional Review Board. All samples were de-identified prior to laboratory testing, and all participants provided informed consent.

The stored plasma specimens that were used in this study had been collected from a convenience sample of potential COVID-19 convalescent plasma (CCP) donors. The donors were recruited in the greater Baltimore, MD and Washington D.C. metropolitan areas from April to December 2020 [4] [5] [6] . Saliva collection was undertaken in this cohort, starting in June 2020.

Individuals were eligible for enrollment if they had a documented history of a positive molecular assay test result for SARS-CoV-2 infection (confirmed by medical chart review or the donor provided clinical documentation) and met standard self-reported eligibility criteria for blood donation. Only individuals who had both plasma and saliva collected on the same day were included in this study (n=108). The study used a complete case analysis approach, whereby five samples with missing values and two that did not pass QC were not used. Thus, 101 paired samples were included in the analysis. The study was cross-sectional and none of the subjects contributed more than one paired saliva / plasma sample. All plasma samples were stored at -80C until testing was performed.

Saliva was collected using the OraSure® Oral Antibody Collection Device (OraSure Technologies, Inc., Bethlehem, PA, USA). The saliva sample was processed according to manufacturer's instructions, which involves adding the saliva contained in the Oral Antibody Collection Device foam paddle into 800 μL of OraSure® sample storage buffer immediately after the collection from participants. All samples were heated to 56°C for 1 hour to inactivate SARS-CoV-2 and stored at -80C until analyzed. Archived pre-pandemic negative saliva samples were collected using the Oracol+ S14 collection device (Malvern Medical Developments, Ltd, Worcester, United Kingdom). These Oracol+ S14 samples were collected in multiple research studies prior to December 2019 and involved adult participants representing a diverse range of sociodemographic characteristics [7] . Pre-pandemic negative saliva samples from prior to December 2019 were also heat-inactivated prior to testing with the multiplex assay.

Plasma specimens were analyzed using three commercially available EIAs: the Euroimmun Anti-SARS-CoV-2 ELISA (Mountain Lakes, NJ), the BioRad Platelia SARS-CoV-2

Total Ab (BioRad Laboratories, Hercules, CA), and the Ortho Vitros SARS-CoV-2 IgG EIA (Ortho-Clinical Diagnostics Inc, Rochester, NY). All EIAs were purchased from the manufacturer and conducted according to the manufacturers' instructions, except for the Ortho Vitros EIA in which plasma-rather than serum as recommended-was used. The BioRad EIA measures IgG, IgM and IgA specific for SARS-CoV-2 nucleocapsid protein, whereas the Euroimmun and Ortho Vitros EIAs only measure anti-SARS-CoV-2 IgG specific to spike (S).

The EIAs results are reported as follows: the Euroimmun EIA provides an arbitrary unit ratio (AU, which is the optical density [OD] of the sample divided by calibrator provided), the Ortho Vitros EIA provides a S/CO ratio, and the BioRad EIA provides an OD. Positive samples were defined using the manufacturers' suggested cutoff.

Quantitation of nAb titers against 100 fifty percent tissue culture infectious doses (TCID50) was performed using a nAb assay, as previously describe [4] . Briefly, two-fold dilutions of plasma were started at 1:20 dilutions; SARS-CoV-2/USA-WA1/2020 virus (BEI Resources) was added and allowed to incubate for one hour at room temperature. The virusantibody mixture was transferred to VeroE6-TMPRSS2 cells in 96 well plates and incubated for 6 hours at 37C. The wells were then washed and incubated an additional 48 hours before being scored for protection from cytopathic effect. The nAb area under the curve (AUC) values were estimated using the exact number of wells protected from infection at every plasma dilution; samples that had no NT activity were assigned an arbitrary value of one-half of the lowest nAb AUC.

The total IgG concentration in each participant's collected saliva (i.e., saliva added to OraSure sample storage buffer) was determined using Salimetrics Salivary Human Total IgG ELISA Kit (Salimetrics, LLC, Carlsbad, CA, USA) according to the manufacturer's instructions with the following modification: the sample incubation and the detection antibody incubation times were reduced to 1 hour instead of 2 hours. This modification was approved by the manufacturer. The high and low assay controls provided by the manufacturer were included on every plate and found to be within the manufacturer-specified ranges.

Saliva samples were tested using a multiplex SARS-CoV-2 IgG immunoassay as previously described, based on Luminex technology [1] . The multiplex assay included magnetic bead sets (MagPlex microspheres) coupled covalently with antigen (5 μg antigen per 1 million beads) [1] . The assay included SARS-CoV-2 nucleocapsid (N), receptor binding domain (RBD), and spike (S) antigens (Supplemental Table 1 ). Briefly, saliva was centrifuged for 5 minutes at 20,000 g. Because all CCP saliva samples were collected using the OraSure® Oral Antibody Collection Device (OraSure Technologies, Inc., Bethlehem, PA, USA), which includes 800 μL of OraSure® sample storage buffer, the saliva contained in the collection device (foam paddle) is diluted (estimated 4-fold dilution compared to our prior testing of Oracol-collected saliva samples). Thus, instead of 10 μL as described previously [1] , 40 μL of this combined saliva / OraSure sample buffer was added to each well along with 10 μL PBST/1% BSA (assay buffer) containing 1,000 beads per bead set for a final volume of 50 μL. Each assay plate contained 1-2 blank wells with OraSure sample buffer instead of samples that were used for background subtraction. We used commercially-available pooled human saliva (Innovative Research, Inc., Michigan, USA), purchased prior to the pandemic, as negative control saliva. A positive control was created by spiking a during-pandemic saliva sample from a SARS-CoV-2 RT-PCR confirmed individual that was highly positive for SARS-CoV-2-specific IgG into a pre-pandemic negative control saliva sample. Because the pre-pandemic saliva samples were collected using the Oracol+ S14 device, which does not contain any sample storage buffer, 10 μL of undiluted saliva was added to each well along with 40 μL PBST/1% BSA (assay buffer) containing 1,000 beads per bead set for a final volume of 50 μL. Phycoerythrin-labeled anti-human IgG diluted 1:100 in assay buffer was used to detect IgG binding to antigens. The plates were read on a Luminex MAGPIX instrument.

For salivary SARS-CoV-2 multiplex assay results, the blank-subtracted (-net‖) median fluorescence intensity (MFI) was used for statistical analyses. Cutoffs to discriminate IgG positive from IgG negative samples for each individual antigen and for combinations of multiple antigens (algorithms) had previously been calculated using the average net MFI plus three standard deviations of n=265 pre-COVID-19 era negative control specimens. The sensitivity and specificity for individual and for combinations of SARS-CoV-2 antigens had been calculated using saliva samples collected >14 days after COVID-19 symptoms onset and pre-COVID-19 era saliva samples as described elsewhere [1] ]. Highest accuracy (98.6% sensitivity [143/145] and 99.2% specificity [263/265] was achieved using a summary index of SARS-CoV-2-specific IgG S/CO to seven N, RBD and S antigens [unpublished data] and by applying a minimum sample quality control (QC) threshold based on total salivary IgG concentration (μg/mL). A total salivary IgG QC threshold was applied to samples that were negative for the summary index of SARS-CoV-2-specific IgG S/CO to seven N, RBD and S antigens. As with pre-pandemic Oracol-collected saliva samples, any OraSure-collected saliva sample containing less than 0.15 μg total IgG per 50-μL assay reaction was considered to not pass sample QC and was excluded from the analysis if the summary index of SARS-CoV-2-specific IgG S/CO to seven N, RBD and S antigens did not cross the cutoff of 6. This quality control measure excludes samples that could potentially be classified as false negatives as a result of improper saliva sample collection or insufficient total salivary IgG concentration.

The concordance of the multiplex salivary SARS-Cov-2 IgG assay with 3 blood-based EIAs (using manufacturer's cutoffs) and nAb titers (using AUC of 20 as cutoff since it is the lower limit of detection of the assay) was examined using positive percent agreement (PPA), negative percent agreement (NPA), percent agreement (PA) and Cohen's kappa coefficient.

Indeterminate and borderline results of Euroimmun and BioRad were considered to be negative. 

Demographic information for the subjects that contributed specimens for this analysis is shown in Supplemental Table 2 (n=101). The median age was 44 years (interquartile range

[IQR]=34-56), 42.6% were male, and the majority of the participants (72.3%) were non-Hispanic White. Only 14.9% were hospitalized due to COVID-19. There was a median of 50 days (IQR=40-70 days) between diagnostic PCR+ assay and sample collection for this study.

In comparison to the three commercial serological EIAs, performance was generally best for the RBD antigens within the multiplex salivary SARS-CoV-2 IgG assay and the Ortho Vitros EIA ( Table 1 ). The highest percent agreement was between the Ortho Vitros EIA and the multiplex salivary SARS-CoV-2 IgG assay's GenScript RBD-specific IgG (PPA=97.7%; NPA=76.9%) and sum of all three RBD-specific IgG S/CO values (PPA=98.9%, NPA=76.9%) ( In comparison to a nAb assay (considering an area under the curve [AUC] <20 as a negative result and ≥20 as a positive result), the best performance was observed for the multiplex salivary SARS-CoV-2 IgG assay's Mt. Sinai whole spike-specific IgG result (PPA=65.2%;

NPA=96.9%). The next best comparative performance with nAb was followed closely by the multiplex salivary SARS-CoV-2 IgG assay's Sino Bio ECD antigen (PPA=97.1%; NPA=25.0%) and the Native Antigen Company (NAC) N and all 3 RBD antigens (PA range: 72.3%-73.3%) ( Table 1) . The concordance between each component of the multiplex SARS-CoV-2 antibody assay with the final algorithmic result in the multiplex salivary assay is presented in Supplemental Table 3 .

The concordance between SARS-CoV-2 plasma antibody tests is shown in Supplemental 1 ). Correlations between SARS-CoV-2 plasma antibody tests are shown in Supplemental Table   6 . Correlations with nAb AUC were better for the Ortho Vitros EIA S/CO (ρ=0.83; 95% CI=0.74, 0.89; p<0.001) and Euroimmun EIA AU (ρ=0.80; 95% CI=0.72, 0.86; p<0.001) than those for the multiplex salivary assay and nAb AUC.

When compared to a S/CO value of 12 or greater using the Ortho Vitros EIA (i.e., the requirement for high-titer designation under the EUA), the receiver operating characteristic (ROC) area under the curve (AUC) for the multiplex salivary assay's sum of S/CO ratios for N/RBD/S antigens was 0.92 (95%CI=0.87, 0.97) (Figure 2 ).

In this study of eligible CCP donors, a multiplex salivary SARS-CoV-2 IgG assay's performance was comparable to three commercially-available SARS-CoV-2 serological tests.

Importantly, as a surrogate of nAb activity, the multiplex salivary SARS-CoV-2 IgG assay appeared to demonstrate equivalent or slightly better percent agreement than the three commercially-available serological EIAs. The range of agreement observed between saliva and plasma assays was wide and depended on the degree of overlap between antigenic targets used in each assay. We observed better comparative performance for antigens in the salivary multiplex SARS-CoV-2 IgG assay that matched the proteins employed by the EIAs, while those targeting different proteins were generally observed to have lower agreement. For example, the Ortho Vitros and Euroimmun EIAs both measure S-specific IgG responses whereas the BioRad EIA measures N-specific IgG responses. This is evidenced by the good agreement for e.g., the sum of Saliva has become an important specimen type for the diagnosis of both active and previous infection with SARS-CoV-2. For one, saliva collection is minimally-invasive compared to other commonly used sampling approaches such as phlebotomy and nasopharyngeal-, midturbinate-, or anterior nares swabs. Saliva has been shown to serve as a robust alternative to nasopharyngeal swabs for SARS-CoV-2 RNA testing, which lends itself to population-level diagnosis and/or surveillance [8] [9] [10] [11] . Specifically, collection can be self-administered without technical expertise or oversight. Salivary SARS-CoV-2 viral load can also serve as a dynamic correlate of COVID-19 severity and mortality [12] . The utility of SARS-CoV-2 RNA detection in saliva has emerged in parallel with proofs-of-principle to detect SARS-CoV-2-specific antibody responses in saliva, which could serve as a surrogate of SARS-CoV-2 serological tests [1, 2, 13, 14] . SARS-CoV-2-specific IgG in saliva reflects the blood-derived transudate in the oral cavity; by contrast, SARS-CoV-2-specific IgA in saliva may represent a localized mucosal IgA response to infection [15] [16] [17] . Because the salivary matrix effects are complex and IgG antibodies are present at lower concentrations than in blood, maintaining the diagnostic accuracy particularly high sensitivity or PPAof a salivary -serological‖ assay relative to blood-based serological assays has proven challenging, particularly when using passive drool compared to more antibody-enriched gingival crevicular fluid swab collection methods. Others have developed salivary SARS-CoV-2 antibody assays with varying accuracy. Some have applied multiplex bead-based assay technology to measure SARS-CoV-2 IgG and IgA in unstimulated saliva, achieving reasonable sensitivity (88%) and optimal specificity (up to 100%) [18] . Others have tested unstimulated saliva samples using commercially-available ELISA kits (e.g., Euroimmun), demonstrating a proof-of-principle for associational risk factor analysis, but not for evaluation of assay threshold-setting and validation performance characteristics in saliva [19] .

Others have developed in-house ELISAs that produce AUC information which is then used to characterize longitudinal SARS-CoV-2 antibody kinetics in saliva over time rather than assay accuracy / performance [2] . Our multiplex assay approach produces robust signals using combinations of multiple SARS-CoV-2 N, RBD, and S antigens within a single saliva sample and facilitates optimization of algorithms that can produce both high sensitivity and specificity [1] (see Supplemental Materials File Methods and Table 7a -7b). Furthermore, the semiquantitative nature of the S/CO values generated by the salivary SARS-CoV-2 multiplex IgG assay could offer insight into factors driving the duration and magnitude of SARS-CoV-2 IgG seropositivity in high risk and general populations [18, [20] [21] [22] [23] [24] . Saliva can be used to investigate SARS-CoV-2 vaccine immunogenicity (i.e., responsiveness), population seroprevalence, and durability of antibody response profiles related to natural SARS-CoV-2 infection and / or vaccination and changing dynamics as variants continue to emerge.

This study has several limitations. First, the sample size was relatively small and included a limited number of samples from known-negative donors (as determined by standard plasma serological and nAb assays). The cross-sectional design precluded evaluation of antibody dynamics over time, particularly given the relatively short period during which sample collection was undertaken (i.e. relative to symptom resolution). Nonetheless, this is consistent with most CCP collections and study populations, which provided insight into SARS-CoV-2 immunopathogenesis and screening options [25, 26] . Second, the study population was primarily focused around Baltimore, MD and Washington DC, thus potentially limiting generalizability of the findings. Third, the Ortho Vitros EIA was validated for serum rather than plasma; however, plasma samples have been previously shown to have reliable performance despite departure from the manufacturer's instructions [27] . Finally, a different saliva collection device (OraSure, Bethlehem, PA, USA) was used to establish the negative threshold for the multiplex SARS-CoV-2 IgG assay (Oracol+ S14, Malvern Medical Developments, Worcester, UK). A larger study is recommended to test reproducibility of the findings following the manufacturer instructions for the assay kits and using the same saliva collection devices. Nonetheless, the study employed a robust multiplex saliva assay approach to detect antibodies against SARS-CoV-2, demonstrating favorable performance both against several commercial EIAs, including the Ortho Vitros, as well as formal viral neutralization.

When applied to saliva, the highly-adaptable multiplex approach enables detection of a diverse range of SARS-CoV-2 antigen-specific IgG responses using a minimally-invasive biospecimen that can be self-collected at home. If the findings are replicated in larger studies and validated in vaccination populations and individuals infected with different variants, this could support a scale-up of SARS-CoV-2 serosurveillance to improve population-scale understanding of the extent of SARS-CoV-2 transmission, identify areas with waning (or gaps in) immunity, and also support monitoring of the magnitude and duration of natural infection in the face of vaccination and emerging variants of concern. This non-invasive approach for sero-surveillance may be particularly useful among children who are not yet approved to receive a SARS-CoV-2 vaccine.

We acknowledge all of the participants who contributed specimens to this study and the study staff without whom this study would not have been possible. We thank Florian Krammer, 

Note: S/CO = signal to cut off ratio. AU = arbitrary units. OD = optical density. nAb = neutralizing antibody. AUC: area under curve. Only the saliva assays having the highest Spearman's correlation coefficients (ρ) with each blood assay were plot. ρ were calculated with 95% confidence intervals (CI) estimated over 1000 bootstrap iterations. The straight vertical black line indicates the cut-off for SARS-CoV-2 positivity of saliva assays. The straight horizontal black line represents the cut-off for SARS-CoV-2 positivity of serology assays. The dashed horizontal line indicates the cut-off for SARS-Cov-2 high antibody titer for Ortho Vitros test. 

Note: a signal to cut-off value of 12 or greater of Ortho Vitros EIA were considered high antibody titer. AUC: area under curve; CI: confidence interval. 95% CI of AUC was computed using DeLong method. 

National Institute of Environmental Health Sciences (NIEHS) grant R01ES026973

Serology at Population Scale: SARS-CoV-2-Specific Antibody Responses in Saliva

Persistence of serum and saliva antibody responses to SARS-CoV-2 spike antigens in COVID-19 patients

Detection of SARS-CoV-2 RNA and Antibodies in Diverse Samples: Protocol to Validate the Sufficiency of Provider-Observed, Home-Collected Blood, Saliva, and Oropharyngeal Samples

Sex, age, and hospitalization drive antibody responses in a COVID-19 convalescent plasma donor population

Evaluation of Serological SARS-CoV-2 Lateral Flow Assays for Rapid Point of Care Testing

Comparative performance of five commercially available serologic assays to detect antibodies to SARS-CoV-2 and identify individuals with high neutralizing titers

Face Mask Use and Persistence of Livestock-associated Staphylococcus aureus Nasal Carriage among Industrial Hog Operation Workers and Household Contacts

Comparison of Saliva and Nasopharyngeal Swab Nucleic Acid Amplification Testing for Detection of SARS-CoV-2: A Systematic Review and Meta-analysis

Simply saliva: stability of SARS-CoV-2 detection negates the need for expensive collection devices. medRxiv : the preprint server for health sciences

Pooling saliva to increase SARS-CoV-2 testing capacity. medRxiv : the preprint server for health sciences

Saliva or Nasopharyngeal Swab Specimens for Detection of SARS-CoV-2. The New England journal of medicine

Saliva viral load is a dynamic unifying correlate of COVID-19 severity and mortality. medRxiv : the preprint server for health sciences

ELISA detection of SARS-CoV-2 antibodies in saliva

Detection of antibodies to the SARS-CoV-2 spike glycoprotein in both serum and saliva enhances detection of infection. medRxiv : the preprint server for health sciences

SARS-CoV-2 identification and IgA antibodies in saliva: One sample two tests approach for diagnosis

Dual lateral flow optical/chemiluminescence immunosensors for the rapid detection of salivary and serum IgA in patients with COVID-19 disease

Salivary anti-SARS-CoV-2 IgA as an accessible biomarker of mucosal immunity against COVID-19. medRxiv : the preprint server for health sciences

Persisting Salivary IgG Against SARS-CoV-2 at 9 Months After Mild COVID-19: A Complementary Approach to Population Surveys

Systemic and mucosal antibody responses specific to SARS-CoV-2 during mild versus severe COVID-19

Decline in SARS-CoV-2 Antibodies After Mild Infection Among Frontline Health Care Personnel in a Multistate Hospital Network -12 States

Immunological memory to SARS-CoV-2 assessed for up to 8 months after infection

Healers Sustain Anti-SARS-CoV-2 Antibody Production

Persistence of Anti-SARS-CoV-2 Antibodies in Non-Hospitalized COVID-19 Convalescent Health Care Workers

Durability of SARS-CoV-2-specific IgG responses in saliva for up to 8 months after infection. medRxiv : the preprint server for health sciences

SARS-CoV-2-specific CD8+ T cell responses in convalescent COVID-19 individuals

Antibody Avidity Responses in COVID-19 Patients and Convalescent Plasma Donors. The Journal of infectious diseases

Comparability of six different immunoassays measuring SARS-CoV-2 antibodies with neutralizing antibody levels in convalescent plasma: From utility to prediction. Transfusion