key: cord-1045897-uyq6urw3
authors: Bryan, Andrew; Pepper, Gregory; Wener, Mark H.; Fink, Susan L.; Morishima, Chihiro; Chaudhary, Anu; Jerome, Keith R.; Mathias, Patrick C.; Greninger, Alexander L.
title: Performance Characteristics of the Abbott Architect SARS-CoV-2 IgG Assay and Seroprevalence in Boise, Idaho
date: 2020-07-23
journal: J Clin Microbiol
DOI: 10.1128/jcm.00941-20
sha: ac58ff51d6313aa6985318a300a40a28d6c8eef0
doc_id: 1045897
cord_uid: uyq6urw3

Coronavirus disease 2019 (COVID-19), the novel respiratory illness caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), is associated with severe morbidity and mortality. The rollout of diagnostic testing in the United States was slow, leading to numerous cases that were not tested for SARS-CoV-2 in February and March 2020 and necessitating the use of serological testing to determine past infections. Here, we evaluated the Abbott SARS-CoV-2 IgG test for detection of anti-SARS-CoV-2 IgG antibodies by testing 3 distinct patient populations. We tested 1,020 serum specimens collected prior to SARS-CoV-2 circulation in the United States and found one false positive, indicating a specificity of 99.90%. We tested 125 patients who tested reverse transcription-PCR (RT-PCR) positive for SARS-CoV-2 for whom 689 excess serum specimens were available and found that sensitivity reached 100% at day 17 after symptom onset and day 13 after PCR positivity. Alternative index value thresholds for positivity resulted in 100% sensitivity and 100% specificity in this cohort. We tested specimens from 4,856 individuals from Boise, ID, collected over 1 week in April 2020 as part of the Crush the Curve initiative and detected 87 positives for a positivity rate of 1.79%. These data demonstrate excellent analytical performance of the Abbott SARS-CoV-2 IgG test as well as the limited circulation of the virus in the western United States. We expect that the availability of high-quality serological testing will be a key tool in the fight against SARS-CoV-2.

C oronavirus disease 2019 (COVID-19) is a novel respiratory illness caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a novel Sarbecovirus that emerged from Wuhan, China, in late 2019 (1) . COVID-19 often progresses to lower respiratory tract illness and can be associated with severe morbidity and mortality (2) .

Serological testing can detect past cases of SARS-CoV-2 for which reverse transcription-PCR (RT-PCR) testing was not performed or for which nasopharyngeal swab sampling resulted in false negatives. Serological tests require exceptional sensitivity and specificity, especially when seroprevalence is low, in order to have adequate positive predictive value (3) . To date, most SARS-CoV-2 serological tests on the market have inadequate performance characteristics to be used for widespread population or clinical testing (4) . Here, we evaluated the Abbott SARS-CoV-2 IgG test for use on the Abbott Architect platform. This assay detects IgG antibodies against the SARS-CoV-2 nucleocapsid protein.

Patient cohorts. Samples for specificity testing were derived from deidentified excess serum specimens sent to our clinical virology laboratory in 2018 and 2019. Samples for sensitivity testing were derived from excess serum specimens sent for clinical testing from persons who tested RT-PCR positive for SARS-CoV-2 during March and April 2020. With the exception of the studies of biologic precision, for patients with an IgG result for more than 1 aliquot on a specific date following onset of symptoms or PCR positivity, only the mean index value for that patient-day was included in the data set to minimize the bias from individual patient seroconversion and variable numbers of samples per patient. For the calculations of sensitivity and specificity at the patient level using the manufacturer's recommended index value cutoff of 1.40 (Fig. 1) , patients were assumed to be seronegative on each day preceding the most recent negative IgG result and to be seropositive on each day following an initial positive result. Serum specimens sent from the Boise, ID, metropolitan area were collected over a 1-week period in late April 2020 as part of the Crush the Curve initiative. This work was approved under a consent waiver by the University of Washington institutional review board.

IgG testing. Serum samples were run on the Abbott Architect instrument using the Abbott SARS-CoV-2 IgG assay after FDA notification following the manufacturer's instructions. The assay is a chemiluminescent microparticle immunoassay for qualitative detection of IgG in human serum or plasma against the SARS-CoV-2 nucleoprotein. The Architect platform requires a minimum of 100 l of serum or plasma. Qualitative results and index values reported by the instrument were used in analyses.

Data analysis and visualization. Patient demographic information (sex and age) was extracted alongside laboratory order and result data (including index value) from the laboratory information system (Sunquest Laboratory, Tucson, AZ). Partial analysis using area under the concentration-time curve (AUC) and data visualization were performed using the R packages pROC, ggplot2, and cowplot (5, 6) .

To determine assay specificity, we used 1,020 deidentified serum specimens from 1,010 different individuals sent to our laboratory for herpes simplex virus (HSV) Western blot serology in 2018 and 2019, before SARS-CoV-2 was thought to be circulating in Washington State and the United States (7). One serum specimen from this set tested positive, with an initial index value of 1.41 and a value on repeat testing of 1.49, above the Abbott-determined positivity cutoff of 1.40 (Fig. 1A ). All other specimens tested negative, leading to an assay specificity of 99.90% in pre-COVID-19 serum.

To determine assay sensitivity, we used serum specimens from a series of 125 patients who tested RT-PCR positive for SARS-CoV-2 and for whom 689 excess serum specimens comprising 415 unique patient follow-up days were available. The vast majority of these patients were hospitalized at the University of Washington Medical Center-Northwest Campus in Seattle, WA, between March and April 2020. Fifty-eight percent of patients were male and 42% female. .5%) at 10 days, 100.0% at 14 days (95.4% to 100.0%), and 100.0% (95.5% to 100.0%) at 17 days using the manufacturer's recommended cutoff of 1.4. Intriguingly, 22 of 88 individuals (25%) for which serum was available on the first day of PCR positivity had simultaneous detection of serum anti-SARS-CoV-2 IgG and nasopharyngeal SARS-CoV-2 RNA (Fig. 1B) .

We next used our SARS-CoV-2 IgG index values over 415 unique patient-days to assess the change in index value over time, from the date of symptom onset (Fig. 1C) and first positive PCR result (Fig. 1D ). For these patients, early in the course of their infections, index values consistently increased over time, on review both of individual patients with multiple IgG results over time and of aggregate summary data.

Based on our data suggesting consistent seroconversion and the low false-positive rate in our specificity study, we next asked what the optimal index value cutoffs were for different days after onset of symptoms or PCR positivity. Partial AUC analysis was performed by setting the minimum specificity between 99.0% and 99.9% (Tables 1 and  2) to minimize false positives, given the low seroprevalence to SARS-CoV-2 expected in our population, and to identify the optimal index thresholds for different potential uses of the test. These analyses indicated that optimal thresholds for the serologic diagnosis of SARS-CoV-2 were 1.42 to 1.49 at Ն17 days from symptom onset (sensitivity and specificity, 100%), 0.7 at Ն14 days from onset (sensitivity, 97.9%; specificity, 99.6%), 0.7 at Ն10 days from onset (sensitivity, 94.4%; specificity, 99.6%), and 0.7 at Ն7 days from onset (sensitivity, 88.0%; specificity, 99.6%) (Fig. 2) .

Given our large unique data set, we next assessed the biologic variation of the antibody results in PCR-positive patients by examining the results of all tests for which at least 3 remnant serum or plasma samples were available from the same day for the same patient. The coefficient of variation was calculated for each of 75 available patient-days and plotted against the index value (Fig. 3A) . The reproducibility of the measurable anti-SARS-CoV-2 IgG response was robust across the index value range except for 2 situations: (i) higher coefficients of variation (CVs) associated with very low index values (i.e., Ͻ0.1) related to analytical "noise" and (ii) higher CVs related to the rapid change in antibody levels associated with active seroconversion. The CV was Ͻ10% for all included patient-days with an index value above 0.4 except for 4 data points representing 3 different patients in the process of seroconversion. For 3 of these 4 patient-days with CVs of Ͼ10%, samples had been drawn several hours apart. To further examine the process of seroconversion in individual patients, we identified 7 patients that had IgG results available on at least 5 different patient-days and for whom we captured the process of seroconversion, plus 1 patient that appeared to be in the process of seroconverting but did not cross the positivity threshold (Fig. 3B ). In addition to assessment of the biologic variation, traditional analytic precision was determined: the same remnant sample at an index value of ϳ2.2 was analyzed 5 to 10 times on each of 3 Abbott Architect instruments, yielding individual CVs between 1.4% and 2.5% and a cumulative CV of 2.6% (cumulative mean, 2.26). We also measured qualitative reproducibility at approximately 20% above and below the index value cutoff of 1.40. All 10 measurements below the cutoff were negative (CV, 1.5%), and those above the cutoff were positive (CV, 1.1%). Those without a reported gender had a seropositivity of 2.6%. Seroprevalence was highest among those over 80 years (4%), 60-to 69-year-olds (2.5%), and 20-to 29-yearolds (2.3%) and was lowest among those under 19 years of age (0.4%).

Here, we report the performance characteristics of the recently available Abbott SARS-CoV-2 IgG assay. Using the manufacturer's recommended index value cutoff of 1.40 for determining positivity, we report an assay specificity of 99.9% from 1,020 pre-COVID-19 serum specimens and sensitivity of 100% at 17 days after symptom onset and 13 days after PCR positivity. Our results mirror that of the assay package insert, which details a 99.6% specificity from Ͼ1,000 specimens presumed SARS-CoV-2negative and 100% sensitivity by day 14 after symptom onset.

In our own cohort, we found that increasing the threshold would have resulted in 100% specificity and 100% sensitivity at 17 days after symptom onset. However, the optimal threshold may depend on the intended clinical use of the test and the characteristics of the target population. Given the limitations of clinical sensitivity of SARS-CoV-2 PCR testing for various sample types, IgG serology with an applied low threshold may be a useful adjunctive diagnostic for patients with negative PCR results who have been symptomatic for Ն7 days with a clinical presentation consistent with COVID-19 disease. In contrast, a higher threshold might be considered for PCR-negative asymptomatic patients for assessing previous undiagnosed infection. For laboratories reporting a single diagnostic result for both populations, it may be useful to report an inconclusive range corresponding to an optical density (OD) ratio of roughly 0.8 to 1.5 with a recommendation for repeat testing to minimize false-negative results associated with seroconversion. At this time, repeat serology may be preferable to a diagnostic algorithm using a secondary assay, as no specific confirmatory assay with sufficient sensitivity and specificity exists. Our serological validation was chiefly limited by the use of excess serum specimens from a mostly hospitalized population known to be very recently infected with SARS-CoV-2. This convenience sample meant that PCR and serology data were not available for each day since symptom onset, requiring us to censor follow-up days accordingly (e.g., days before if the first longitudinal serological result was positive or days afterward if the last serological result was negative). The majority of patients in this study were elderly individuals-65.6% were older than 60 years of age-many of whom also had altered mental status at time of presentation, complicating our ability to accurately ascertain symptom onset. The elderly, hospitalized population used in our sensitivity cohort could account for the delayed time to positivity seen in our cohort versus the Abbott package insert (17 versus 14 days after symptom onset), as declining immune responses are associated with advanced age (8) . It is unclear what the prevalence of antibody is in individuals with subclinical or asymptomatic infections and how this assay performs in an asymptomatic population. We were also restricted to limited descriptive epidemiological information on the serological survey conducted within the Boise, ID, metropolitan area. Given the self-selected nature of this cohort of persons interested in their SARS-CoV-2 serostatus, we expect that our measures of seropositivity in Boise likely overestimate the true seroprevalence of the virus in this community. The Abbott SARS-CoV-2 IgG test is also limited in that it detects only IgG antibodies directed against nucleocapsid and cannot be used for recombinant spike protein vaccine studies.

Overall, our data demonstrate excellent performance of the Abbott Architect SARS-CoV-2 IgG assay and a high level of consistency with the package insert. Our data reinforce the limited circulation of SARS-CoV-2 in the Pacific Northwest during early 2020. We expect that high-quality serological testing will be an important component of the diagnostic approach to SARS-CoV-2.

A novel coronavirus from patients with pneumonia in China

Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observational study

Serological assays for emerging coronaviruses: challenges and pitfalls

Double-edged spike: are SARS-CoV-2 serologic tests safe right now

pROC: an open-source package for R and Sϩ to analyze and compare ROC curves

ggplot2: elegant graphics for data analysis

Cryptic transmission of SARS-CoV-2 in Washington State

Causes, consequences, and reversal of immune system aging

We thank the team behind Crush the Curve Idaho for facilitating sample collection and delivery for SARS-CoV-2 IgG testing. We also thank the University of Washington Medical Center Northwest Campus clinical laboratory staff for reserving remnant serum and plasma samples from COVID-19 PCR-positive patients, particularly Leanne Gilly and Michelle Manis. This work was supported by the Department of Laboratory Medicine at the University of Washington Medical Center.A.L.G. reports personal fees from Abbott Molecular, outside the present work.