key: cord-0746225-8d0x5prs
authors: Endo, Akira
title: Estimated Sensitivity Values of Severe Acute Respiratory Syndrome Coronavirus 2 Tests from Cross-sectional Data Warrant Caution Due to Unvalidated Model Assumptions
date: 2020-10-26
journal: Clin Infect Dis
DOI: 10.1093/cid/ciaa1656
sha: b5bdfc7cea0a52326b429e3d24004f9b4de54eec
doc_id: 746225
cord_uid: 8d0x5prs

nan

To the Editor-I read with interest the recent study by Yokota et al [1] on polymerase chain reaction (PCR) tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) between nasopharyngeal swab (NPS) and saliva samples. The authors used a Bayesian latent-class model to simultaneously estimate the sensitivity and specificity of PCR tests along with the unknown prevalence in the study cohorts (contact tracing and airport screening), and reported a sensitivity of 86% and 92%, respectively, for NPS and saliva samples and over 99.9% specificity for both tests. They also reported high concordance between them and concluded that saliva samples can replace the conventional NPS. Compared with NPSs, saliva tests are low-invasive and require less involvement of healthcare professionals. The equivalent performance of saliva tests, also suggested in other studies [2] [3] [4] [5] , has a profound impact on the ongoing public health planning against coronavirus disease 2019 (COVID-19) where efficient and safe testing protocols are of paramount importance.

However, I noticed that the authors' model has limitations due to the following 2 conditions, which rendered the reported sensitivity of 86% and 92% merely reflective of positive agreement rates (PARs) [6] between NPS and saliva tests.

• NPS and saliva test results are assumed to be independent, although the authors showed that their cycle threshold (Ct) values are highly correlated (with a Kendall's W [7] of 0.87). • The prevalence p was jointly estimated with other parameters, which caused the sensitivity almost solely determined by PAR in the data.

I showed that posterior distributions of PAR obtained from the original data are, in effect, identical to those of sensitivity obtained from the full dataset: NPS, 86% (90% credible interval, 77-93%); saliva, 90% (82-96%) ( Figure 1A ). Almost identical distributions were also obtained from data excluding individuals testing negative for both tests ("negativenegatives"; 1872/1924 participants): NPS, 88% (79-94%); saliva, 91% (83-96%). These indicate that the original sensitivity estimates almost solely relied on individuals receiving at least 1 positive result (47 from contact tracing and 5 from airport screening data) and that the rest of the dataset was barely informative, which was caused by the latent variable p freely optimized reflecting PARs and the independent assumption. By using simulation, I also showed that, in the presence of high correlation between (hypothetical) viral loads in NPS and saliva samples (Pearson's correlation, 0.8), I can reproduce the observed contact-tracing data and Kendall's W (0.86), which nonetheless suggest lower sensitivity (62% for NPS and 67% for saliva) than the original study ( Figure 1B and 1C) .

These results highlight the inherent limitations of the original study relying on potentially intercorrelated single-point PCR tests without external validation of true infection statuses. Sensitivity and PAR correspond under the independence assumption, and this study estimated sensitivity based on that assumption. While I support the finding of concordance between NPS and saliva tests, the interpretation of their sensitivity estimates warrants caution. In practice, multiple factors can lead to viral loads of both NPS and saliva below the detectable limit, including samples taken too early or late in the infection course or simply of poor quality. Polymerase chain reaction will remain the de facto gold-standard test; the possibility of a lessinvasive sampling method without apparent loss in performance is an invaluable finding. However, any tests should be used with a proper understanding of their limitations in performance-for example, how frequently and, more importantly, when they can fail-which this study was unfortunately not sufficient to inform.

Code availability. Detailed methods and replication codes are available on GitHub (https:// github.com/akira-endo/reanalysis_Yokota2020).

Financial support. A. E. is supported by The Nakajima Foundation.

Potential conflicts of interest. A. E. received a research grant from Taisho Pharmaceutical Co., Ltd. The author has submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

Mass screening of asymptomatic persons for SARS-CoV-2 using saliva

Saliva as a noninvasive specimen for detection of SARS-CoV-2

Saliva sample as a non-invasive specimen for the diagnosis of coronavirus disease 2019: a cross-sectional study

Self-collected oral fluid and nasal swabs demonstrate comparable sensitivity to clinician collected nasopharyngeal swabs for Covid-19 detection. medRxiv

Saliva or nasopharyngeal swab specimens for detection of SARS-CoV-2

Enabling precision medicine with digital case classification at the point-of-care

The problem of m rankings