key: cord-0738163-bu2uz3y5
authors: Henry, Brandon Michael; Benoit, Stefanie W; Lippi, Giuseppe; Sabalza, Maite; Venkataraman, Iswariya; Geisler, Daniel; Freeman, Megan Culler; Rapsinski, Glenn J; Wheeler, Sarah E
title: False-Positive Rates in Pediatric SARS-CoV-2 Serology Testing: Biological, Clinical, and Technical Considerations
date: 2021-09-04
journal: Am J Clin Pathol
DOI: 10.1093/ajcp/aqab118
sha: 4856bfe9929c6ed2eaf159ec9f4e88534a987bb1
doc_id: 738163
cord_uid: bu2uz3y5

nan

Indeed, although RBD-binding antibodies are commonly referred to as "neutralizing" antibodies, perfect equivalence between RBD-binding antibodies' neutralizing antibodies and the actual neutralizing potential of serum or plasma does not exist. For example, a recent study found that the concordance among 7 anti-SARS-CoV-2 immunoassays and virus-neutralization tests varied widely between a minimum of 0.24 and a maximum of 0.72. 5 It may also be that anti-SARS-CoV-2 antibodies may differ in that decay of more specific anti-RBD antibodies may precede decay of generic anti-S1 (or S1/2) antibodies. This behavior would make it challenging to define whether anti-RBD antibodies are false negatives or if anti-S1 antibodies are false positives. These considerations are important when performing serology studies, with anti-S1/S2 serology tests preferably used to assess true seroprevalence; anti-RBD-specific serology immunoassays may be better suited to reflecting the presence of neutralizing antibodies.

Finally, it is noteworthy that the EUROIMMUN Anti-SARS-CoV-2 IgG assay has been well validated in the current scientific literature; it was the first serology assay to be independently validated by the Frederick National Laboratory for Cancer Research. That said, we encourage all diagnostic companies to provide pediatric-specific diagnostic performance data and support the independent validation and publication of their assay performance in pediatric populations by academic medical centers.

Ultimately, more studies are required to determine the false positivity rate in children using SARS-CoV-2 serologic assays. Clinically characterized samples are critical to such conclusions, while differences in the technology, antigen used, signal detection (ie, enzyme-linked immunosorbent assay [ELISA] vs chemiluminescence), and detected immunoglobulin classes are essential when interpreting test results. 

We at EUROIMMUN have read the article published by Geisler et al 1 and would like to provide our response to it. Despite the interesting study, including a cohort of pediatric patients, we believe that the authors have misleadingly settled on the title "Unexpected False-Positive Rates in Pediatric SARS-CoV-2 Serology Using the EUROIMMUN Anti-SARS-CoV-2 ELISA IgG Assay." After reviewing the publication, we disagree with the authors' result interpretations and would like to address and highlight the various reasons for it. First, it is important to consider that the samples investigated in this study were collected during the coronavirus disease 2019 (COVID-19) pandemic. It has been estimated that 16% to 50% of the pediatric infections are asymptomatic. 2 Therefore, it is possible that one could expect few positive results in the cohort tested. Additionally, only samples that were positive by EUROIMMUN assay were retested using the other commercial assays. The evaluation of all the samples with all assays would have been preferred to avoid underestimating results and to be able to compare the overall positivity rates across manufacturers.

As the authors reported, it is important to note that there are differences among the 4 assays in terms of the antigen target, measured immunoglobulin classes, and detection technology implemented. Therefore, one would expect to find inconsistencies in the results. The EUROIMMUN assay incorporates the full-length S1 protein subunit, including the receptor-binding domain (RBD), while the other assays incorporate only the RBD. Compared with the RBD moiety, more epitopes are available on the full-length S1 subunit.

The results of the study showed a positive rate of 3.64% (95% CI, 2.91%-4.48%) with the EUROIMMUN assay. This rate translates into a specificity of 96.36%, which is greater than the 95% criterion that the US Food and Drug Administration accepted. This high specificity rate is not proportionate to the high false positivity rate (80.82%) that the authors reported. The performance of our enzyme-linked immunosorbent assay (ELISA), including its excellent specificity, has been confirmed by several external studies conducted to date. 3 Additionally, our assay was the first serology assay independently validated by federally funded research (the National Cancer Institute). The results reported excellent sensitivity and specificity for the EUROIMMUN assay. 4 Recently, another external research institute validated our assay, which resulted in inclusion of our assay in the list of tests for use in the manufacture of COVID-19 convalescent plasma. 5 During CE validation of EUROIMMUN assay, we also tested a set of 100 pediatric samples of children aged 3 to 10 years (samples collected before 2020). Only 1 sample tested weak positive, which corresponds to an overall specificity of 99% (unpublished data).

In conclusion, the study by Geisler et al 1 does not accurately represent the true specificity of the EUROIMMUN Anti-SARS-CoV-2 IgG ELISA. We ask the editors to reevaluate this publication for appropriate interpretation of the results and to address additional limitations of this study as highlighted above. We agree on the need for more studies with pediatric samples to improve the accuracy of serology tests, but it is important to include clinically characterized samples for these studies to draw appropriate conclusions.

EUROIMMUN US, Mountain Lakes, NJ, USA r e F e r e n C e s

We thank the authors for their interest in our work and appreciate the further discussion of our data and study limitations. For severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), it has been and will be critical to determine whether children are exposed to the virus to inform public health decisions, particularly as many children remain ineligible for vaccination. Because of a lack of published data on test performance in pediatric patients, assessment of serologic testing using multiple approaches is critical for proper test use and clinical interpretation. As the authors indicated, larger and more comprehensive studies are needed. Pretest sensitivity and specificity of assays are critical performance metrics that compare a cohort of known positive with known negative patients. All assays used in this work have independently tested sensitivity and specificity of more than 98% in our own and other adult studies. These assays have demonstrated excellent concordance when compared with the assay consensus of 6 clinical tests using receptor-binding domain or S1 protein antigens. 1 In our study, we had sufficient remnant specimen for follow-up testing for 73 of the 85 initially positive samples from a total cohort of 2,338 specimens, which limits our analysis. Limited sample volume is common with residual clinical samples, however, particularly in pediatrics. 2 Serology plays a key role in seroprevalence studies, the case definition for multisystem inflammatory syndrome in children, a reportable disease, and in determining prior SARS-CoV-2 infection. In clinical settings, we are interested in the posttest probabilities calculated with the validation sensitivity/specificity and disease prevalence. Typical performance characteristics assessed are the false-positive rate (FPR) and the positive predictive value (PPV) or their negative complements, false-negative rate, and negative predictive value. 3 In vaccinated populations, seroprevalence resulting from vaccination is quickly rising, but in pediatric cohorts, prevalence is often still quite low. At the time of this study, the estimated pediatric prevalence based on adult seroprevalence was approximately 1%. 4 At a prevalence of 1% in a serosurvey of 2,338 children and an assay with 99% sensitivity and specificity, 50% of our positive results are calculated to be false positives. 5 When performing orthogonal testing in our study, the first test screens a low-prevalence population (1%). The second test, with identical sensitivity and specificity, will be screening a population with an expected 50% prevalence, based on the anticipated true-positive rate of the first test. Therefore, on this principle, one would calculate no false positives or false negatives by the second test because of the enriched prevalence. Any minor change in specificity (because of population differences), however, will lead to large changes in the FPR. At a 1% prevalence, with a sample size of 2,338 and a sensitivity of 99%, we calculate that specificities of 99%, 98%, 97%, 96%, and 95% would result in FPRs of 50%, 67%, 75%, 80%, and 83%, respectively. These data are not a criticism of the tests themselves, but they remind us of the importance of orthogonal testing, particularly in a low-prevalence population. Small changes in specificity caused by lack of assay characterization in a population can also alter FPR and PPV. Arguably, the most straightforward way to manage this alteration is to incorporate a second test of all positive samples, which increases the prevalence for the second test, and our findings support this recommendation. As we all note, more work is needed to fully understand assay performance and immune response to SARS-CoV-2 in children.

Unexpected false-positive rates in pediatric SARS-CoV-2 serology using the EUROIMMUN Anti-SARS-CoV-2 ELISA IgG assay

Evaluation of the EUROIMMUN Anti-SARS-CoV-2 ELISA assay for detection of IgA and IgG antibodies

Silver Spring, MD: US Food and Drug Administration

Letter to Nikki Bratcher-Bowman regarding tests acceptable for use in the manufacture of high titer COVID-19 convalescent plasma

MD: US Food and Drug Administration

Variable performance in 6 commercial SARS-CoV-2 antibody assays may affect convalescent plasma and seroprevalence screening

Unexpected false-positive rates in pediatric SARS-CoV-2 serology using the EUROIMMUN Anti-SARS-CoV-2 ELISA IgG assay

Predictive values, uncertainty, and interpretation of serology tests for the novel coronavirus

Diagnostic test calculator. Alan Schwartz's Web site