key: cord-1008345-05xterl2
authors: Schuler, Charles F.; Gherasim, Carmen; O’Shea, Kelly; Manthei, David M.; Chen, Jesse; Giacherio, Don; Troost, Jonathan P.; Baldwin, James L.; Baker, James R.
title: Accurate point-of-care serology tests for COVID-19
date: 2021-03-16
journal: PLoS One
DOI: 10.1371/journal.pone.0248729
sha: 3c7a92555110b3cd30d8b72bc674b20b078266ae
doc_id: 1008345
cord_uid: 05xterl2

BACKGROUND: As COVID-19 vaccines become available, screening individuals for prior COVID-19 infection and vaccine response in point-of-care (POC) settings has renewed interest. We prospectively screened at-risk individuals for SARS-CoV-2 spike and nucleocapsid protein antibodies in a POC setting to determine if it was a feasible method to identify antibody from prior infection. METHODS: Three EUA-approved lateral flow antibody assays were performed on POC finger-stick blood and compared with serum and a CLIA nucleocapsid antibody immunoassay. Variables including antibody class, time since PCR, and the assay antigen used were evaluated. RESULTS: 512 subjects enrolled, of which 104 had a COVID-19 history and positive PCR. Only three PCR-positive subjects required hospitalization, with one requiring mechanical ventilation. The POC results correlated well with the immunoassay (93–97% sensitivity) and using serum did not improve the sensitivity or specificity. CONCLUSIONS: Finger-stick, POC COVID-19 antibody testing was highly effective in identifying antibody resulting from prior infections in mildly symptomatic subjects. Using high-complexity serum immunoassays did not improve the screening outcome. Almost all individuals with COVID-19 infection produced detectable antibodies to the virus. POC antibody testing is useful as a screen for prior COVID-19 infection, and should be useful in assessing vaccine response.

a1111111111 a1111111111 a1111111111 a1111111111 a1111111111

Coronavirus disease 2019 , caused by severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), has caused a pandemic with millions of cases and deaths [1] . As effective vaccines become available, the question of screening individuals for prior COVID-19 infection has become relevant. Serological testing is the best accepted method to detect prior COVID-19 infection. In contrast to polymerase chain reaction testing, which is positive during acute infection, serologic testing identifies antibody responses to prior infection [2, 3] . Antibody assays have been useful for prevalence determinations; however, fuller validation of serologic testing to detect COVID-19 infection is still needed. Current assays use different SARS-CoV-2 detection antigens and have different testing conditions [2] [3] [4] . This is important since all current vaccines and those in development only involve spike protein. In addition, some assays identify both virus-specific IgM and IgG while others only identify IgG. The most efficient approach to screen for antibody is in point-of-care (POC) settings, but no assay is approved for this use.

CLIA laboratory-based antibody assays are expensive, require blood draws for serum, and need sample transport to a CLIA-approved facility. Rapid antibody tests using lateral flow assay(s) (LFA) solve these logistical problems but have had other issues. While initially loosely regulated their function has now been verified by a standardized EUA in the US requiring a prior FDA technical review. Despite this, the accuracy of these assays has continued to be questioned, and no test is currently approved for finger-stick POC use [5, 6] . Thus, real-world data are needed on LFA accuracy to identify prior COVID-19 in practical use; we therefore aimed to evaluate three COVID-19 LFA tests in a POC setting.

In this study, we prospectively evaluated the ability of LFA to detect antibody in a cohort at risk for COVID-19. We used LFA from three manufacturers who had obtained full EUA approval after evaluation by the FDA. We tested subjects for antibody in a POC setting and compared the POC LFA results with results from serum obtained at the same time that was analyzed separately. LFA results were also compared to a high-moderate CLIA laboratory immunoassay run on the same serum sample and compared to the subjects' clinical history and PCR results. The results suggest potential utility for LFA to screen for prior COVID-19 infection in the POC setting.

This study was reviewed and approved by the U-M Institutional Review Board (IRB) and conducted according to the principles expressed in the Declaration of Helsinki. All participants provided informed written consent. Enrollment occurred between April 28, 2020 and October 2, 2020. All study activities occurred at the University of Michigan (U-M) Division of Allergy and Clinical Immunology, Department of Clinical Pathology, and Biomedical Science Research Building. All subjects were recruited via phone or email announcements with phone follow-up. Potential subjects were selected who had recently had a COVID-19 PCR test (see below) were contacted by phone. U-M healthcare workers were recruited via approved emails to work areas impacted by COVID-19 patient care.

Three populations were recruited. Exclusion criteria included immunodeficiency/immunosuppression with known primary or acquired immunodeficiency; anti-rejection therapy following solid organ or bone marrow transplant; biologic therapeutics such as tumor necrosis factor inhibitors; known malignancy and chemotherapy; systemic immunosuppressive therapy, including corticosteroids equivalent to 20 mg/day of prednisone for 2 weeks.

Data were collected in Research Electronic Data Capture software (REDCap, Vanderbilt University). Subjects reported demographics, past medical history, and medications. For known COVID-19-positive subjects, the infection course, travel history, known exposure to COVID-19, and, if applicable, the hospital course, including medications, laboratory and imaging, intensive care needs, mechanical ventilation, and complications, were reviewed. Any other illnesses that subjects had during the time-period after March 1 st , 2020 were reviewed. The subjects' medical records were also reviewed to verify key data.

Capillary blood samples by finger-stick were obtained with spring-loaded lancets using aseptic technique. Blood samples of 10 mL were collected via standard phlebotomy using aseptic technique into no-additive collection tubes and refrigerated at 4˚C until same-day processing. Samples were centrifuged at 2000 g for 10 minutes at 4˚C. Serum was transferred into 2 mL aliquots in glass screw-thread vials (DWK Life Sciences) and stored at -20˚C until analysis.

POC COVID-19 antibody tests were used from Access Bio (CareStart COVID-19 IgM/IgG or "CareStart"), Autobio Diagnostics Co Ltd (Anti-SARS-CoV-2 Rapid Test or "Autobio") and Healgen Scientific (COVID-19 IgG/IgM Rapid Test Cassette or "Healgen"). The CareStart detects both SARS-CoV-2 spike (receptor binding domain or RBD portion) and nucleocapsid antibodies, while the Autobio and Healgen detect SARS-CoV-2 spike (RBD) antibodies (Healgen detects S1 protein antibodies). Each test was run per each company's EUA-Instructions for Use [7, 8] , except testing was done with both whole blood from finger-stick and serum from a blood draw and incubated as appropriate. Tests were considered positive if IgM and/or IgG were positive. A faint line was read as positive.

During the course of the study the Autobio had its EUA revoked related to poor performance of the IgM portion of the test; we stopped using this device at the request of our IRB [8, 9] . We also evaluated several other LFA that did not have FDA EUA approval. However, preliminary analysis of these assays showed performance that was inadequate to include in the results.

Nucleocapsid antibody testing was performed on the Elecsys1 (Roche) SARS-CoV-2 Total Antibody Assay ("immunoassay"), run on a Cobas e411 analyzer in the Clinical Core Laboratories at U-M, a CLIA-certified lab. The assay detects total SARS-CoV-2 nucleocapsid (N) antibodies via a sandwich electrochemiluminescence assay (ECLIA). Positive and negative controls were included with each run. A cutoff index (COI) of > 1 is used to report a positive result. The assay detects all SARS-CoV-2 N antibodies, including IgG [8] . This assay has been well-characterized by our institution previously [10] .

Descriptive statistics were provided using medians and interquartile ranges (for continuous variables) and frequencies and percentages (for categorical variables). Finger-stick assay results were compared to serum results, immunoassay, and PCR; serum results were also compared to the immunoassay and PCR. For finger-stick and serum, separate analyses were done by brand and antibody (i.e., IgM and/or IgG). 2x2 contingency tables frequencies and measures of diagnostic accuracy (i.e., sensitivity, specificity, positive and negative predictive value) were calculated for each comparison. Area under the curve (AUC) was calculated using logistic regression. Time between antibody test and PCR were compared between antibody positive and negative tests using an unpaired t-test. Two-sided α = 0.05 was used to assess statistical significance, and no adjustments were made for multiple comparisons as there were relatively few (by brand, by IgM vs. IgG, and by serum vs. POC) and were considered distinct comparisons. Analyses were based on complete-case analyses; participants with missing data for a given analysis were excluded. No outlier values for the variables included were observed (most data elements are "positive vs. negative"). Subgroup variation in agreement by age and occupation (HCW vs. general population) was tested using interaction terms in logistic regression models. 512 participants were enrolled. Assuming 15% would be COVID positive, and assuming a true AUC of 0.9, this sample would have 80% power to detect a confidence interval width of 0.09, with two-sided alpha = 0.05. Analyses were performed in SAS V9.4 (SAS Institute Inc., Cary, NC, USA) and Prism V8.0 (GraphPad Inc., San Diego, CA, USA).

We enrolled 512 subjects; 104 had PCR-confirmed COVID-19. 93 subjects had a known negative COVID-19 PCR within 14 days of enrollment. The remainder had no history of a PCR. No subjects had a known COVID-19 reinfection. The median age was 39 years old, 75% were female, and 88% were healthcare workers (Table 1) . Among PCR-confirmed COVID-19 subjects, symptoms included fever >38˚C (53%), subjective fever (77%), chills (79%), myalgias (82%), headache (75%), diarrhea (58%), anosmia (70%), dysgeusia (64%), rhinorrhea (47%), sore throat (52%), cough (77%), dyspnea (56%), nausea and/or vomiting (35%), and abdominal pain (30%). Three of the COVID-19 positive cases were hospitalized; one required ICU care and mechanical ventilation (Fig 1) .

We assessed LFA performance using subject serum and compared these to PCR and the immunoassay. CareStart showed 96.2%/89.2% sensitivity/specificity with PCR and 98.3%/ 94.0% sensitivity/specificity with the immunoassay (Table 2) . Healgen showed 95.2%/88.2% sensitivity/specificity with PCR and 96.5%/94.2% sensitivity/specificity with the immunoassay (Table 2) . Autobio showed a lower sensitivity/specificity, 87.8%/84.6% with PCR and 93.2%/ 96.1% with the immunoassay (Table 1 ) over 149 subjects tested prior to EUA revocation. Notably, among subjects with a negative or no prior PCR, 10 (5%) were positive on CareStart, 11 (6%) were positive on Healgen, and 2 (4%) were positive on Autobio (Fig 2A-2C, Table 2 ).

These LFA give separate IgG and IgM results. In general, using only IgG results for the LFA produced slightly better sensitivity/specificity results for all three tests, detailed in Table 3 . Further, IgM should appear soon after initial infection and decay over time. Among PCR-positive subjects, we plotted time between PCR and LFA using serum against IgM result. For CareStart, there was a trend toward a shorter interval from positive PCR among those with a positive IgM (Fig 2D) . For Healgen, there was no difference in time between PCR and LFA testing for IgM status (Fig 2E) . For Autobio, IgM positive subjects had a significantly shorter time between PCR and LFA (Fig 2F) .

We compared LFA performance using finger-stick blood and serum (Fig 3A-3C) . The Car-eStart showed 96.6%/98.9% sensitivity/specificity between finger-stick whole blood and separated serum and 98.1%/93.4% sensitivity/specificity between finger-stick and the immunoassay ( Table 2 ). The Healgen showed 92.1%/97.0% sensitivity/specificity agreement between fingerstick and serum and 93.2%/94.9% sensitivity/specificity between finger-stick and the immunoassay ( Table 2 ). The Autobio showed 88.4%/97.9% sensitivity/specificity between finger-stick and serum and 90.5%/98.0% sensitivity/specificity between finger-stick and the immunoassay ( Table 2) . As above, all three LFA showed slightly improved test characteristics using fingerstick blood when IgG was considered alone (Table 3 ). There was no significant subgroup variation in agreement by age or employment. The CareStart serum vs. POC comparison showed slightly higher agreement among older individuals (AUC = 0.95 among those <40; = 0.99 among those 40+, p = 0.03); all others tests showed no difference by age. We also evaluated LFA IgM results among PCR positive subjects using finger-stick. CareStart showed no difference in time between PCR and LFA testing (Fig 3D) . Healgen showed significantly more IgM positive subjects when assayed a shorter time between PCR and LFA testing ( Fig 3E) ; the Autobio assay showed a similar but non-significant trend (Fig 3F) .

The COVID-19 pandemic has profoundly affected society [11, 12] , causing millions of cases and deaths [1] . With the advent of effective vaccines [13] , identifying individuals with prior COVID-19 has renewed interest. Importantly, knowing prior COVID-19 infection history could help in vaccine prioritization and evaluating individuals with prior infections and analyzing for vaccine responses and side effects [14, 15] . Despite this, there is a debate whether SARS-CoV-2 antibody can serve as a surrogate for COVID-19 immunity, which remains unresolved in part due to poor testing methods [14, 16] . This was especially true of POC antibody assays, which were initially released without adequate FDA review. While this problem has In this prospective study, we demonstrate that FDA EUA-approved COVID-19 antibody LFA can provide accurate measures of prior COVID-19 infection; this holds true even when the tests are used with finger-stick blood. There are several unique aspects of this study. The population differs from other studies in that most subjects had a mild infection course with few requiring hospitalizations (Fig 1) . Indeed, we found that COVID-19 antibody production was an accurate marker of prior infection and noted good concordance between PCR status, IgG identified with the immunoassay, and the LFA using either finger-stick whole blood or serum (Tables 2 and 3 ). Nearly all subjects with a positive COVID-19 PCR made detectable antibody, which contrasts with studies suggesting less severe infections may not lead to robust antibody responses [17, 18] .

The CareStart and Healgen assays showed good concordance using serum-based testing and the immunoassay. There was also good agreement between finger-stick blood and serum analyses for these two assays. This suggests that in this partially unselected population, use of an LFA can detect prior unknown COVID-19 infection to a degree similar to a high-moderate complexity immunoassay. The reasons for the improved performance of antibody screening in this study are likely multiple. We used assays that were fully vetted by the FDA before EUA approval. The Autobio assay we examined that had the EUA revoked had less sensitivity compared to the other tests [9] . In addition, all the tests we employed used spike protein as the primary antigen (Access Bio uses both spike and nucleocapsid proteins) and it has been suggested that nucleocapsid assays have less sensitivity.

This study does have some important limitations. First, nearly 90% of subjects were healthcare workers, somewhat limiting the broader applicability of this data. Second, the Autobio 

Point-of-care COVID-19 serology tests sample size was lower than the study sample total due to discontinuing use related to the EUA revocation. Third, one concern with these assays is the false positive rate and specificity, as a false positive could conceivably lead to a false sense of security toward future infection. This could adversely impact risk-taking behavior if the user believes a positive test confirms immunity to future infection, which is not yet known with certainty. One potential source of false positives lies in the concern that pre-existing conditions, such as autoimmune disease or non-SARS-CoV-2 coronaviruses could lead to cross-reactivity and false positives; this is somewhat mitigated by the work required by the FDA for EUA applications and data on the immunoassay from our own institution [8, 10] . Fourth, we did not include immunocompromised subjects, so the applicability of this testing to that population is not known. Finally, cost differences for lab-based testing versus LFA may significantly vary by location and country. In our local area, the clinical cost of a CLIA-lab-based immunoassay is approximately $42 -$119 versus $30 -$50 for an LFA. The LFA cost is expected to decrease in the future. There remains debate whether SARS-CoV-2 antibodies can serve as a surrogate for COVID-19 immunity, which remains unresolved in part due to poor testing methods [14, 16] . This was especially true of POC antibody assays, which were initially released without minimal FDA review. The test that lost EUA during the course of our study demonstrates that effective corrective action has been taken. These data suggest that regulatory agencies are now appropriately approving these tests but may not indicate that all LFA assays are accurate when using finger-stick blood [19] . Importantly, data are emerging that individuals with antibodies against COVID-19 may be at lower risk of infection [20] . The tests evaluated herein correlate well with higher complexity serum antibody assays, require less infrastructure, and can be performed in POC setting without a venous blood draw. This could allow LFA to provide accurate antibody results in primary care and POC situations and assist in prioritizing individuals receiving COVID-19 vaccines. Such a "screen before vaccine" approach might be helpful in prioritizing vaccines in the general population, where those who produce a positive POC test could conceivably wait for a vaccination. Finally, these assays might also serve to screen for antibody responses to the COVID-19 vaccines, as the antibody concentrations generated after vaccination appear to be substantial [7] ; further work would be needed to address this as well.

Writing -original draft: Charles F. Schuler, IV, Jesse Chen, Jonathan P. Troost, James L.

Baldwin, James R. Baker, Jr.

WHO. WHO Coronavirus Disease (COVID-19) Dashboard 2021

SARS-CoV-2 and COVID-19: The most important research questions

Accuracy of serological testing for SARS-CoV-2 antibodies: first results of a large mixed-method evaluation study

Effectiveness of tests to detect the presence of SARS-CoV-2 virus, and antibodies to SARS-CoV-2, to inform COVID-19 diagnosis: a rapid systematic review

The Rapid Coronavirus Antibody Test: Can We Improve Accuracy? Front Med (Lausanne)

Comparative assessment of multiple COVID-19 serological technologies supports continued evaluation of point-of-care lateral flow assays in hospital and community healthcare settings

Durability of Responses after SARS-CoV-2 mRNA-1273 Vaccination

Authorized Serology Test Performance

Revocation of EUA200349 www.fda.gov2020

Differences in Performance Characteristics Among Four High-Throughput Assays for the Detection of Antibodies Against SARS-CoV-2 Using a Common Set of Patient Samples

Disease Control, Civil Liberties, and Mass Testing-Calibrating Restrictions during the Covid-19 Pandemic

Psychosocial impact of COVID-19

UK authorises Pfizer/BioNTech COVID-19 vaccine [webpage]. www.gov.uk: United Kingdom

Immunological considerations for COVID-19 vaccine strategies

COVID-19 herd immunity: where are we?

Immune Response, Inflammation, and the Clinical Spectrum of COVID-19

Differences of SARS-CoV-2 serological test performance between hospitalized and outpatient COVID-19 cases

The dynamics of immune response in COVID-19 patients with different illness severity

Evaluation of three immunochromatographic tests for rapid detection of antibodies against SARS-CoV-2

Antibody Status and Incidence of SARS-CoV-2 Infection in Health Care Workers

The authors acknowledge the efforts of Mariam Farida, our study coordinator. The authors acknowledge Bridgette Kaul and Ann Wegmann, both of whom provided significant assistance with subject recruitment.