key: cord-0785108-81cyze92
authors: Richard-Greenblatt, Melissa; Ziegler, Matthew J; Bromberg, Valerie; Huang, Elizabeth; Abdallah, Hatem; Tolomeo, Pam; Lautenbach, Ebbing; Glaser, Laurel; Kelly, Brendan J
title: Quantifying the Impact of Nasopharyngeal Specimen Quality on SARS-CoV-2 Test Performance
date: 2021-05-12
journal: Open Forum Infect Dis
DOI: 10.1093/ofid/ofab235
sha: 68a9ca8a26eaef5a923ccb914c948f7d0bc381fd
doc_id: 785108
cord_uid: 81cyze92

BACKGROUND: The SARS-CoV-2 reverse-transcription polymerase chain reaction (RT-PCR) cycle threshold (Ct) has been used to estimate quantitative viral load, with the goal of targeting isolation precautions for individuals with COVID-19 and guiding public health interventions. However, variability in specimen quality can alter the Ct values obtained from SARS-CoV-2 clinical assays. We sought to define how variable nasopharyngeal (NP) swab quality impacts clinical SARS-CoV-2 test sensitivity. METHODS: We performed amplification of a human gene target (β-actin) in parallel with a clinical RT-PCR targeting the SARS-CoV-2 ORF1ab gene for 1282 NP specimens collected from patients with clinical concern for COVID-19. We evaluated the relationship between NP specimen quality, characterized by late Ct values for the human gene target β-actin Ct, and the probability of SARS-CoV-2 detection via logistic regression, as well as the linear relationship between SARS-CoV-2 and β-actin Ct. RESULTS: Low quality NP swabs are less likely to detect SARS-CoV-2 (odds ratio 0.607, 95%CI 0.487 to 0.753). We observed a positive linear relationship between SARS-CoV-2 and β-actin Ct values (slope 0.181, 95%CI 0.097 to 0.264), consistent with a reduction in detection of 0.181 cycles for each additional cycle of the β-actin target. COVID-19 disease severity was not associated with β-actin Ct values. CONCLUSIONS: Variability in NP specimen quality significantly impacts the performance of clinical SARS-CoV-2 assays, and caution should be taken when interpreting quantitative SARS-CoV-2 Ct results. If unrecognized, low quality NP specimens, which are characterized by a low level of amplifiable human DNA target, may limit the successful application of SARS-CoV-2 Ct values to direct infection control and public health interventions.

A c c e p t e d M a n u s c r i p t 4

As the COVID-19 pandemic continues to drive morbidity and mortality around the world, interest has grown in using SARS-CoV-2 reverse-transcription polymerase chain reaction (RT-PCR) cycle threshold (Ct) values as a means of quantifying viral load [1, 2] . It has been proposed that SARS-CoV-2 Ct values may correspond with viral burden and infectivity, and that SARS-CoV-2 values may be used to predict disease severity and guide isolation precautions for individuals with COVID-19 [3] [4] [5] [6] [7] . SARS-CoV-2 Ct values have been shown to correspond with community COVID-19 burden, and it has also been proposed that community Ct values may help to guide non-pharmaceutical interventions to control COVID-19 [8, 9] .

We sought to understand the impact of nasopharyngeal (NP) specimen swab quality on the measurement of SARS-CoV-2 Ct and the sensitivity of virus detection. To collect an NP swab for SARS-CoV-2 testing, healthcare workers are instructed to advance a synthetic fiber swab with plastic or wire shaft through the nostril until contacting the posterior nasopharynx at a depth equal to the distance from the nostril to the opening of the ear, then to rub and roll the swab, leaving the swab in place for several seconds to collect secretions, before rotating the swab further as it is removed from the nostril [10] . Variability in practice and patient tolerance of the procedure has been observed and may impact the sensitivity of SARS-CoV-2 detection, as well as the cycle threshold (Ct) value observed when SARS-CoV-2 is detected [11] [12] [13] .

To measure variability in the quality of NP swab collection, we performed amplification of a human gene target (β-actin) in parallel with RT-PCR targeting the SARS-CoV-2 ORF1ab gene. High β-actin Ct values have been previously validated as a marker of poor NP swab quality [2, 14] . Below we report the relationship between quality of NP swab collection, sensitivity of SARS-CoV-2 detection, and the range of impact we expect sub-standard NP swab collection may exert on SARS-CoV-2 Ct values. We also examine the possibility of A c c e p t e d M a n u s c r i p t 5 confounding by greater NP epithelial cell damage associated with increasing COVID-19 disease severity.

Study Design, Setting, and Population: We performed a retrospective cohort study of adult patients presenting to the PennMedicine health system between March 26 and July 4, 2020. The patient population included drive-through symptomatic, asymptomatic preprocedure, and hospital inpatient transfer and discharge patients. As the goal of the study was to evaluate the impact of specimen quality on SARS-CoV-2 assay sensitivity, we included only NP specimens processed by a laboratory-developed emergency use authorization (EUA) SARS-CoV-2 assay that included a human gene target (β-actin) as a specimen quality control [2, 14] . No other SARS-CoV-2 assays in the health system included a human gene target. All specimens were assayed for β-actin and MS2 phage RNA in parallel with SARS-CoV-2. We included all SARS-CoV-2 RT-PCR results for which the positive PCR control analyte (MS2 phage RNA) was detected in the target Ct range (20) (21) (22) (23) (24) (25) cycles) [15] . A total of 1282 NP specimens were included.

A waiver of informed consent was granted by the University of Pennsylvania Institutional Review Board (IRB protocols #843085 & #843274).

We hypothesized that β-actin and SARS-CoV-2 Ct values are related because poor NP specimen collection technique results in reduced capture of NP epithelial cells and SARS-CoV-2 alike. β-actin is a commonly used endogenous reference gene, used as an internal control for PCR reactions involving human specimens. This gene has been previously validated as a marker for the presence of nasal epithelial cells, and prior research has supported its use to assess the quality of self-collected midturbinate swabs [2, 14] . We additionally considered the possibility of confounding by COVID-19 severity of illness A c c e p t e d M a n u s c r i p t 6 because those with more severe infection may have greater NP epithelial cell damage, resulting in greater detection of both PCR targets, irrespective of sampling technique.

To evaluate the possibility of confounding by disease severity, we measured two independent markers of respiratory illness: (1) the minimum room-air oxygen saturation recorded within 2 days of SARS-CoV-2 testing, and (2) whether infiltrates were observed chest computerized tomography (CT) imaging performed within 7 days of SARS-CoV-2 testing. Per Centers for Disease Control and Prevention (CDC) guidelines [16] , we considered room-air oxygen saturation < 94% indicative of severe respiratory illness. We chose to use CT imaging rather than x-ray imaging to discriminate findings likely to be related to SARS-CoV-2. Radiology reports for CT imaging that described parenchymal lung disease, including "infiltrates", "pneumonia", "groundglass", or other "opacities", were considered indicative of severe respiratory illness. The presence of lung nodules, lung masses, chronic airway disease including bronchiectasis, emphysematous changes, or pleural effusions in the absence of parenchymal disease as described above, were not considered indicative of severe acute respiratory illness. We chose to include CTs performed within 7 days of SARS-CoV-2 testing to capture disease severity that could plausibly have confounded the relationship between β-actin and SARS-CoV-2 Ct values.

Specimen Collection, Processing, and RT-PCR Assay: Specimens were collected during routine clinical practice using a nylon flocked mini-tip swab collected in VTM or saline [17] [18] [19] [20] . Healthcare providers obtained samples using CDC guidelines for NP samples collection.

Samples were transported to the laboratory at ambient temperature and stored at 4⁰C if not run immediately. All specimens were analyzed at the Hospital of the University of Pennsylvania's Clinical Microbiology Laboratory using an EUA-approved laboratorydeveloped SARS-CoV-2 assay on the BD MAX system. The multiplex assay was designed to include a specimen quality control (β-actin) [2, 14] , an internal processing control to monitor the RT and PCR steps (MS2 phage RNA) [15] , and a SARS-CoV-2 target (Orf1ab). Exk were used for the lab-developed SARS-CoV-2 assay based on a previously described assay [21, 22] . The BD MAX system was set to run type 1 workflow. PCR conditions consisted of a reverse transcriptase step (600s at 58⁰C, 1 cycle), denaturation step (60s at 98⁰C, 1 cycle) and extension steps (10s at 98⁰C followed by 40s at 63⁰C, 40 cycles). Ct values for all three targets (β-actin, MS2 phage RNA) were recorded. The acceptable Ct range for β-actin in our assay was 24-37. Samples that were negative for SARS-CoV-2 and either negative for βactin or with a β-actin Ct > 37 were reported as invalid.

The primary exposure of interest was the β-actin Ct value, a surrogate for the quality of NP swab collection. The primary outcome of interest was SARS-CoV-2 Ct value.

Statistical Methods: Data were organized using R statistical software version 3.6.1 [23] , and plots generated using the "ggplot2" package [24] . Where β-actin and SARS-CoV-2 were not detected, Ct values were imputed as 40 cycles. We examined (1) the linear relationship between β-actin and SARS-CoV-2 Ct values, as well as (2) the impact of β-actin Ct on SARS-CoV-2 detection using Bayesian linear and generalized-linear mixed effects models, which were fit using Stan Hamiltonian Monte Carlo (HMC) version 2.21, via the "brms" package with default weakly-informative priors [25, 26] . Mixed effects models were fit in the same manner, with random slopes and intercepts according to disease severity (assessed by radiography or oxygen saturation) to evaluate the impact of disease severity on the relationship. Prior predictive modeling was performed, and models were fit with 4 chains of 1000 iterations, confirmed with HMC diagnostics (no divergent iterations, Rhat statistic < 1.1 for all parameters, and E-BFMI > 0.2) [27] [28] [29] . We examined parameter distributions at 50%, 80%, and 95% posterior credible intervals to understand the relationship between exposure and outcome variables. The posterior credible intervals estimate the probability that the true parameter value lies within the interval, based on the observed data.

A c c e p t e d M a n u s c r i p t 8 Power and Sample Size: We estimated the necessary cohort size based on the anticipated effect of poor NP swab quantity [30] . We anticipated that approximately 800 subjects would detect a 10% reduction in sensitivity of SARS-CoV-2 detection related to a β-actin Ct increase of 10, with credible intervals precision ensuring type S error < 5% [31, 32] . We targeted enrollment of 10% more subjects to allow for a margin of error in that estimate, and we exceeded our enrollment target. Table 1 summarizes the SARS-CoV-2 Ct values, as well as those of the specimen quality control, βactin, and RT-PCR reaction control, MS2 RNA. Supplemental Figure 1 depicts the distribution of each target and the relationships between their distributions.

We evaluated the relationship between NP specimen quality, measured by β-actin Ct value, and SARS-CoV-2 test sensitivity with logistic regression, and we found that increasing β-actin Ct values are significantly associated with reduced detection of SARS-CoV-2 (odds ratio 0.607, 95%CI 0.487 to 0.753). Figure 1 shows the relationship between β-actin and SARS-CoV-2 detection probability. We further evaluated the linear relationship between β-actin and SARS-CoV-2 Ct values with linear regression, and we found that SARS-CoV-2 Ct increases significantly with β-actin Ct (slope 0.181, 95%CI 0.097 to 0.264). A linear model restricted to A c c e p t e d M a n u s c r i p t 9 include only the 131 specimens within which both SARS-CoV-2 and β-actin were detectable (i.e., Ct < 40) also found that SARS-CoV-2 Ct increased with β-actin, but this relationship did not have high posterior certainty (slope 0.200, 95%CI -0.295 to 0.720), and linear model fit was poor.

To understand the potential impact of poor NP specimen quality, we evaluated the change in probability of SARS-CoV-2 detection as β-actin Ct increases. We found that a 4-Ct increase in β-actin, from Ct of 28 to Ct of 32 (roughly from the first quartile of observed β-actin Ct values to the third quartile) results in a 6.34% (95%CI 3.35% to 9.17%) decreased probability of SARS-CoV-2 detection.

Threshold: Considering the possibility that the observed association between SARS-CoV-2 and β-actin Ct values is confounded by respiratory illness severity, we evaluated the relationship between β-actin Ct and independent markers of respiratory illness. Oxygen saturation data were available for 425 (33.1%) subjects; chest CT imaging was available for 108 (8.4%) subjects. At least one measure of disease severity was available for 38 (28. persists across all levels of respiratory illness, whether assessed by oxygen saturation or chest radiography (Figure 2) . These analyses of independent markers of severe respiratory disease suggest that it is NP specimen quality, not disease severity, that drives the association between SARS-CoV-2 and β-actin Ct values. To further corroborate these findings, we repeated the analysis, comparing specimens collected from inpatient (n = 858) A c c e p t e d M a n u s c r i p t 10 versus outpatient (n = 424) locations. We found the same association between β-actin Ct values and probability of SARS-CoV-2 detection across both specimen collection sites.

In this study, we found that higher β-actin Ct values, which have been previously validated as a marker of low NP swab quality [2, 14] , were associated with reduced probability of SARS-CoV-2 detection and with higher SARS-CoV-2 Ct values. This finding has several important implications. First, the correlation between β-actin Ct and SARS-CoV-2 suggests that quantitative interpretation of SARS-CoV-2 human specimens may be significantly impacted by the quality of specimen collection, potentially limiting the ability to compare results. In this cohort, we found that each increase in β-actin Ct was associated with an increased SARS-CoV-2 Ct of 0.181. The quantitative interpretation of SARS-CoV-2 RT-PCR Ct results may be enhanced by adjusting for the β-actin Ct. Such adjustment could be used in studies comparing results of serial testing within subjects or populations, where there is a change in β-actin Ct between specimens. Second, the data support the concern that poor specimen collection may contribute to false-negative results. The concern of false-negative NP SARS-CoV-2 testing has led to the recommendation to retest patients with moderate to high clinical suspicion of COVID-19 [33, 34] . Reporting the β-actin Ct, or a β-actin-adjusted SARS-CoV-2 Ct may allow clinicians to better interpret specimen quality when considering retesting.

We considered the possibility that the observed relationship between SARS-CoV-2 and βactin Ct values might be confounded by respiratory disease severity, but we found no significant association between independent markers of severe respiratory disease and lower β-actin Ct values. However, several limitations of our analysis must be acknowledged.

Measures of disease severity are imprecise, and oxygen saturation data and chest CT radiography reports were only available for a small percentage (32.6% and 8.5%, A c c e p t e d M a n u s c r i p t 11 respectively) of our subjects. Subject demographics and medical comorbidities could not be ascertained for subjects, so unmeasured confounders may contribute to the observed association. The impact of comorbid conditions, immune compromise, and duration of illness on the observed relationship are important areas for future study.

Despite our study's limitations, we believe that the observed association between NP specimen quality and SARS-CoV-2 RT-PCR sensitivity is an important finding. From 1282 NP specimens submitted for SARS-CoV-2 testing, we have quantified the variation in specimen quality measured by β-actin Ct value, and we have defined the impact of the observed variation on test sensitivity and SARS-CoV-2 Ct values. The original CDC SARS-CoV-2 assay included a human gene internal control, ribonuclease P (RNAse P). However, most commercial SARS-CoV-2 assays do not include a human gene target. Our findings suggest that caution is required in the clinical interpretation of Ct values from such assays.

SARS-CoV-2 Ct values have shown promise as a means to roughly quantify viral burden and so to guide infection control and public health interventions [1, 2, [4] [5] [6] [7] [8] [9] . However, variability in NP specimen collection may exert large effects on observed SARS-CoV-2 Ct values, limiting these useful applications. As testing efforts expand, infrastructure to ensure quality sample collection must expand as well [10] . Concurrent measurement of a β-actin human gene target may provide a means to recognize and adjust for variability in NP specimen quality.

A c c e p t e d M a n u s c r i p t 12   Tables:   Table 1 : 

Temporal dynamics in viral shedding and transmissibility of COVID-19

Euro surveillance: bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin 2020

SARS-CoV-2 viral load predicts mortality in patients with and without cancer who are hospitalized with COVID-19

Duration of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infectivity: When is it safe to discontinue isolation? Clinical infectious diseases: an official publication of the Infectious Diseases Society of

Clinical infectious diseases: an official publication of the Infectious Diseases Society of

Clinical infectious diseases: an official publication of the Infectious Diseases Society of

Challenges and controversies to testing for COVID-19

Estimating epidemiologic dynamics from single cross-sectional viral load distributions

Using viral load and epidemic dynamics to optimize pooled testing in resource-constrained settings

A direct comparison of enhanced saliva to nasopharyngeal swab for the detection of SARS-CoV-2 in symptomatic patients

Interpreting diagnostic tests for SARS-CoV-2

Sensitivity of chest CT for COVID-19: Comparison to RT-PCR

SARS-CoV-2 nasopharyngeal swab Testing-False-Negative results from a pervasive anatomical misconception

Development and evaluation of a flocked nasal midturbinate swab for self-collection in respiratory virus infection diagnostic testing

Use of bacteriophage MS2 as an internal control in viral reverse transcription-PCR assays

Discontinuation of Transmission-Based precautions and disposition of patients with COVID-19 in healthcare settings (interim guidance)

Comparison analysis of different swabs and transport mediums suitable for SARS-CoV-2 testing following shortages

Evaluation of transport media and specimen transport conditions for the detection of SARS-CoV-2 by use of Real-Time reverse Transcription-PCR

Copan eNAT transport system to address challenges in COVID-19 diagnostics in regions with limited testing access

Evaluation of saline, Phosphate-Buffered saline, and minimum essential medium as potential alternatives to viral transport media for SARS-CoV-2 testing

Real-time PCR-based SARS-CoV-2 detection in canadian laboratories

Comparison of copan ESwab and FLOQSwab for COVID-19 diagnosis: Working around a supply shortage

R: A language and environment for statistical computing

Elegant graphics for data analysis

A probabilistic programming language

An R package for bayesian multilevel models using stan

Statistical rethinking: A bayesian course with examples in R and stan. 1 edition

Visualization in bayesian workflow

Planning study size based on precision rather than power

Type S error rates for classical and bayesian single and multiple comparison procedures

Beyond power calculations: Assessing type S (sign) and type M (magnitude) errors. Perspectives on psychological science: a journal of the Association for

Repeat COVID-19 molecular testing: Correlation of SARS-CoV-2 culture with molecular assays and cycle thresholds

Infectious diseases society of america guidelines on the diagnosis of COVID-19

A c c e p t e d M a n u s c r i p t 14