key: cord-0835706-m7f33ndc
authors: Toubat, O.; Berg, A. H.; Sobhani, K.; Mulligan, K.; Hori, A. M.; Bhattacharya, J.; Sood, N.
title: Manufacturer Signal-to-Cutoff Threshold Underestimates Cumulative Incidence of SARS-CoV-2 Infection: Evidence from the Los Angeles Firefighters Study
date: 2021-04-23
journal: nan
DOI: 10.1101/2021.04.20.21255829
sha: 86bf16b321e2fb29310078d7b9912afbe2b0e9a3
doc_id: 835706
cord_uid: m7f33ndc

While SARS-CoV-2 serologic testing is used to measure cumulative incidence of COVID-19, appropriate signal-to-cut off (S/Co) thresholds remain unclear. We demonstrate S/Co thresholds based on known negative samples significantly increases seropositivity and more accurately estimates cumulative incidence of disease compared to manufacturer-based thresholds.

Designing an appropriate public health response to the pandemic requires an accurate estimate of the cumulative incidence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. While serologic tests for antibodies against SARS-CoV-2 have served as the primary method for modeling the cumulative incidence of disease, factors including test kit sensitivity, waning antibody levels, and disease severity can influence seropositivity estimates (1) (2) (3) . Another important consideration when determining serostatus is the signal-to-cut off (S/Co) threshold used in antibody assays to define seropositive cases. For many commonly used platforms, manufacturer recommended S/Co thresholds were established by testing hospitalized COVID-19 patients, or those with severe illness. Although these thresholds have performed well in populations of similar disease severity, their performance when applied to the general population of those infected with SARS-CoV-2 remains unclear.

We conducted serologic testing to assess the prevalence of SARS-CoV-2-specific antibodies in a cohort of firefighters with a known infection history. We compared the performance sensitivity and specificity of manufacturer recommended S/Co values with modified S/Co values as determined by testing known pre-pandemic negative control samples.

We hypothesized that alternate S/Co thresholds would improve performance characteristics and more accurately estimate cumulative incidence of disease in this cohort of firefighters with mild sickness representative of the majority of SARS-CoV-2 infected individuals.

The Los Angeles County Department of Public Health Institutional Review Board approved this study. We obtained written informed consent from all study participants. We All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 23, 2021. ; https://doi.org/10.1101/2021.04.20.21255829 doi: medRxiv preprint collected data on participant demographics and polymerase chain reaction (PCR)-confirmed SARS-CoV-2 infection history through electronic surveys.

We collected EDTA plasma venipuncture samples from participants who did not report symptoms on the sample collection day. Serology testing was conducted at Cedars-Sinai Medical Center's CLIA-certified laboratory with FDA EUA approvals, using the Abbott Architect SARS-CoV-2 assays for IgM and IgG antibodies against spike and nucleocapsid proteins (Abbott Laboratories, Chicago, IL). We classified participants as being seropositive based on two different thresholds: the manufacturer's recommended S/Co threshold (≥1.4 for IgG; ≥1.0 for IgM) and lower S/Co thresholds based on a validation study that measured antibody levels in 178 negative control patients who had blood drawn prior to the emergence of COVID-19. We summarized descriptive statistics for the study and negative control cohort. We calculated the sensitivity and specificity for manufacturer and modified S/Co thresholds and used percent correctly classified to identify the optimal S/Co threshold based on these samples.

Overall, 585 firefighters that received an antibody test and had a previous PCR test for SARS-CoV-2. Of these, 52 (8.9%) reported having a PCR-positive test history at a median 3.3 months (Interquartile range [IQR], 1.2-4.2 months) prior to antibody testing. Table 1 presents demographic characteristics for individuals with a known history of infection. Most firefighters were male (90.4%) and between 31-59 years of age (88.5%). The most commonly identified racial/ethnic categories in this cohort included White (48.1%) followed by Hispanic (26.9%), Asian (11.5%), Black or African American (7.7%), and Other (5.8%). There were no firefighters that reported being previously hospitalized for the treatment of COVID-19, indicating that this All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 23, 2021. (Table 2) . We found that a modified S/Co of 0.36 yields optimal sensitivity and specificity performance (percent correctly classified=97.0%) in this cohort (Table 2) . When applying this modified threshold to estimate overall serostatus in firefighters, 47 (90.4%) individuals with a previous PCR-positive test result were determined to be seropositive.

Serologic tests can be used to identify active and resolved SARS-CoV-2 infections, which is critical for the epidemiological tracking of COVID-19 in the population. Manufacturer recommended S/Co thresholds for serologic tests are typically based on samples collected from All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 23, 2021. ; https://doi.org/10.1101/2021.04.20.21255829 doi: medRxiv preprint symptomatic patients within a few weeks of infection. While these cut-offs have been shown to perform well in cohorts of similar disease severity, it has been suggested that these thresholds may underestimate cumulative incidence when applied to individuals with milder disease severity or as antibodies wane overtime (2, 3) . Previous analysis of the same Abbott Architect instrument utilized in this study shows that manufacturer based S/Co thresholds were highly sensitive when applied to samples collected from hospitalized patients assayed within 17 days of PCR positivity (4). However, owing to the highly selective characteristics of the sensitivity cohort evaluated in their study, the authors speculate that modified S/Co thresholds may be considered for diagnostic serology in different target populations (4) .

To this end, we evaluated the performance of the Abbott Architect using manufacturer recommended S/Co thresholds in a cohort of firefighters with a known history of SARS-CoV-2 infection. In contrast to previous studies evaluating the performance characteristics of this instrument, none of the firefighters included in the present analysis were hospitalized for the treatment of COVID-19. Our data demonstrate that the sensitivity of manufacturer S/Co thresholds significantly underperforms in ambulatory patients, particularly with increasing time since initial PCR positive test. As a result, we pursued alternate S/Co thresholds based on IgGantibody titer index values measured in known negative samples collected from pre-COVID-19 serum. Analysis of the percent correctly classified according to multiple alternate S/Co thresholds determined the optimal S/Co threshold to be 0.36. The sensitivity and specificity of this threshold was 90.4% (95% CI, 79.0-96.8%) and 99.4% (95% CI, 96.9-99.9%), respectively.

When applied to the firefighter cohort, this threshold improved the overall seroprevalence estimate by 19.2% compared to the manufacturer recommended S/Co threshold. Given that approximately 85% of individuals infected by SARS-CoV-2 are not expected to be hospitalized All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 23, 2021. ; https://doi.org/10.1101/2021.04.20.21255829 doi: medRxiv preprint for the management of their disease, our data suggest that lowering S/Co thresholds from manufacturer recommended levels may be more optimal for antibody assessments in the general population (5) .

Overall, our data demonstrate that the use of a modified S/Co based on known negative samples significantly increases the percent seropositive and more accurately estimates cumulative incidence of infection in a cohort with a disease severity largely representative of the general COVID-19 population. These results are consistent with evidence from studies from US and Belgium which find detectable antibodies several months after infection (6, 7) . In summary, our findings suggest that estimation of cumulative incidence of COVID-19 using serology-based assays must apply diagnostic thresholds that account for weaker antibody response exhibited by those with mild disease.

The findings of this study should be viewed in light of its limitations. First, seroprevalence was assessed based on blood specimens drawn at a single, cross-sectional time point, resulting in varying times since initial PCR positivity. Future serology studies evaluating S/Co thresholds may benefit from repeated measurements to longitudinally track antibody kinetics over time. In addition, PCR positivity was determined by participant survey, thus we cannot rule out false-positive PCR test histories. Finally, it is unclear whether factors such as age, race/ethnicity, concomitant comorbidities, and cross-reactivity with other known coronaviruses influenced SARS-CoV-2 antibody measurements in the negative control cohort utilized in this study. Additional studies evaluating SARS-CoV-2 antibody levels in larger and more representative negative control specimens are needed.

All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 23, 2021. ; https://doi.org/10.1101/2021.04.20.21255829 doi: medRxiv preprint 

All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 23, 2021. ; https://doi.org/10.1101/2021.04.20.21255829 doi: medRxiv preprint

Rodríguez-Barraquer I. Are Seroprevalence Estimates for Severe Acute Respiratory Syndrome Coronavirus 2 Biased? The Journal of Infectious Diseases

Rapid Decay of Anti-SARS-CoV-2 Antibodies in Persons with Mild Covid-19

Longitudinal observation and decline of neutralizing antibody responses in the three months following SARS-CoV-2 infection in humans

Performance characteristics of the Abbott Architect SARS-CoV-2 IgG assay and seroprevalence in

Hospitalization and mortality associated with SARS-CoV-2 viral clades in COVID-19

Persistence of IgG response to SARS-CoV-2. The Lancet Infectious Diseases

Robust neutralizing antibodies to SARS-CoV-2 infection persist for months

All rights reserved. No reuse allowed without permission.(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint this version posted April 23, 2021. ; https://doi.org/10.1101/2021.04.20.21255829 doi: medRxiv preprint