key: cord-304656-v0fyb161 authors: Balayla, J.; Lasry, A.; Gil, Y.; Volodarsky-Perel, A. title: Prevalence Threshold and Temporal Interpretation of Screening Tests: The Example of the SARS-CoV-2 (COVID-19) Pandemic date: 2020-05-22 journal: nan DOI: 10.1101/2020.05.17.20104927 sha: doc_id: 304656 cord_uid: v0fyb161 The curvilinear relationship between a screening test's positive predictive value (PPV) and its target disease prevalence is proportional. In consequence, there is an inflection point of maximum curvature in the screening curve defined as a function of the sensitivity and specificity beyond which the rate of change of a test's PPV declines sharply relative to disease prevalence. Herein, we demonstrate a mathematical model exploring this phenomenon and define the prevalence threshold point where this change occurs. Understanding where this prevalence point lies in the curve has important implications for the interpretation of test results, the administration of healthcare systems, the implementation of public health measures, and in cases of pandemics like SARS-CoV-2, the functioning of society at large. To illustrate the methods herein described, we provide the example of the screening strategies used in the SARS-CoV-2 (COVID-19) pandemic, and calculate the prevalence threshold statistic of different tests available today. This concept can help contextualize the validity of a screening test in real time, thereby enhancing our understanding of the dynamics of the current pandemic. The novel SARS-CoV-2 (COVID-19) has reached over 4 million confirmed cases 54 worldwide; in the United States alone, the death toll totals close to 90,000 to date [1] . 55 Despite efforts to contain its spread, the number of confirmed cases has continued to rise. 56 These estimations have been largely based on polymerase chain reaction (PCR) detection 57 of actively replicating viral material, found only among individuals who are actively 58 infected at the time of testing. Those who recover from the virus without having been 59 tested during the window of viral shedding are not included in prevalence estimates; 60 neither are asymptomatic individuals, roughly approximated as 18 to 30% of those 61 infected [2, 3] . Current estimates thus understate the true prevalence of COVID-19 by 62 failing to include the aforementioned groups. Whereas accurate prevalence estimates are 63 needed to inform public health measures, screening methods that depend on actively 64 replicating viral organisms will become less reliable as recovery proceeds and the 65 resulting prevalence of active infection among the population tapers. Bayes theorem thus 66 demonstrates the point beyond which the positive predictive value (PPV) of currently 67 employed PCR screening will decline as the curve 'flattens' over time. In contrast, 68 serologic testing for COVID-19 antibodies persist in plasma well beyond the period of 69 active infection; as recovery from active infection increases, so too will the cumulative 70 prevalence of seropositivity among the population over time. Herein, we use differential 71 equations to assess the geometry of screening curves and aim to describe the prevalence 72 threshold point beyond which the PPV of various COVID-19 screening tests declines 73 most acutely. Though this example is specific to the COVID-19 pandemic, the methods 74 herein described apply to all screening tests whose sensitivity and specificity parameters 75 are known. 76 The validity of a screening test is defined as the ability to correctly delineate 78 individuals who have a given disease or condition from those who do not. The following 79 four parameters are used to assess the validity of screening tests: sensitivity a, specificity 80 b, positive predictive value φ, and negative predictive value σ. Sensitivity refers to the 81 proportion of people with a given disease who test positive for said disease, also termed 82 the true positive rate. Specificity, also termed true negative rate, refers to the proportion 83 of people without said disease who indeed test negative. Sensitivity and specificity are 84 properties inherent to the screening test itself and are unaffected by the prevalence of 85 disease in a given population. On the other hand, the positive predictive value is defined 86 as the percentage of patients with a positive test that do in fact have the disease, and 87 conversely, the negative predictive value refers to the percentage of patients with a 88 negative test that do not have the disease. These two parameters are dependent upon the 89 prevalence of disease being studied. Using Bayes' theorem, we can derive the following 90 equation expressing positive predictive value p(φ) as a function of disease prevalence φ. 91 93 where: 94 φ = prevalence, p(φ) = positive predictive value, a = sensitivity and b = specificity 95 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020. . https://doi.org/10. 1101 Bayes' theorem describes the probability of an event, based on prior knowledge of 97 conditions that might be related to the event. The relationship between a screening test's 98 positive predictive value, p(φ), and its target disease prevalence φ is proportional -though 99 not linear in all but one case where the sum of sensitivity and specificity equals one. 100 From this curvilinear relationship, as stipulated in Figure 1 , we can derive the prevalence 101 threshold φ e at which the sharp inflection point in the screening curve occurs, as depicted 102 by the following equation (derivation available as a supplement). 103 Additionally, the COVID-19 RT-PCR was tested against 30 respiratory microorganisms, 114 which yielded a specificity nearing 100%. While the above are analytical properties of 115 the COVID-19 RT-PCR test, estimates of its clinical properties vary significantly 116 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020. . https://doi.org/10.1101/2020.05.17.20104927 doi: medRxiv preprint sampling by health care workers (wrong angle, too fast to withdraw and swab, etc.), 118 varying viral load and the subclinical stage when screening is carried out (colonization, 119 incubation, prodrome or acute infection). We will therefore use the FDA's published 120 estimates for simplicity, where φ e = prevalence threshold for detection of COVID-19, a = 121 sensitivity = 0.95, b = specificity = 0.99 and a + b = 1.94. Thus, as per equation 2, the 122 prevalence threshold for RT-PCR detection of COVID-19 is calculated as follows: 123 In graphic form, the screening curve depicts the PPV as a function of prevalence as such: 126 127 The vertical line in red represents the prevalence threshold φ e at 0.093 (9.3%). 128 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020 . . https://doi.org/10.1101 are likely lower than those estimated analytically by the FDA; the prevalence threshold 130 for detection that we have estimated above is thus likely conservative. Nonetheless, the 131 implication is such that below a prevalence of 9.3% actively replicating cases of COVID-132 19 in the population, the PPV of RT-PCR testing declines almost exponentially. In other 133 words, we would expect a sharp increase in the number of false positive screening tests, 134 in turn falsely increasing the estimated prevalence of disease. Even with a conservative 135 estimation of the prevalence threshold, there are potentially significant health, social, and 136 economic implications of false positive screening tests. 137 Serology testing for SARS-CoV-2 involves blood-based testing for antibodies to 139 COVID-19. This screening method identifies all groups of individuals sub-acutely 140 infected or recovered from COVID-19, including those who may be asymptomatic at the 141 time of testing. Use of this tool can thus provide public health officials with a more 142 reliable estimation of the spread of COVID-19 and its cumulative prevalence among 143 different populations over time. Furthermore, this information can bring forth attempts at 144 a better understanding of disease transmission and immunity, which regarding COVID-145 19, is largely uncertain up until this point. 146 The Johns Hopkins Center for Health Security recently released a report outlining 147 the sensitivity and specificity of various serology tests approved for diagnostic use in the 148 United States [5] . These are summarized in Table 1 , with prevalence thresholds estimated 149 for each test. In contrast to RT-PCR screening, COVID-19 testing with serology can 150 delineate immune individuals at a prevalence threshold as low as 4.3%. As the 151 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020 . . https://doi.org/10.1101 positive COVID-19 IgG screen can be reliably accepted as a true positive. In contrast, the 153 nasal RT-PCR test is most reliable when over 9.3% of the population has actively 154 replicating virus at any given time -a value thankfully thus far not reached, even after 155 accounting for an excess positive cases that are not tested. 156 Several countries have already started employing serology-based testing, either 157 for research purposes or to grant 'immunity certificates' to those in whom antibodies to 158 COVID-19 are found [6] . However, we must bear in mind that just as with RT-PCR 159 testing, serologic testing has inherent limitations related to the lag between acute 160 infection and the development of IgM and IgG antibodies as well. In the United States, 161 the Centers for Disease Control and Prevention (CDC) recently released a COVID-19 162 Serology Surveillance Strategy [7] . This strategy aims to employ serology testing, termed 163 'seroprevalence,' at the large-scale (ie. highly impacted areas such as New York and 164 Washington), the community-scale (with systematic selection of participants) and the 165 small-scale (specific subgroups, eg. healthcare workers). From the reasoning above, we 166 conclude that the best screening method to assess for acute infectivity, the need for 167 isolation, and the ensuing social and economic repercussions, is nasal swab RT-PCR for 168 actively replicating virus. In contrast, the most reliable screening method to assess for 169 disease prevalence and burden of disease is antibody testing via serology -assuming 170 enough time has lapsed since the acute infectious episode. 171 In an effort to expedite testing speed and availability, rapid diagnostic tests (RDT) 173 have been developed for point-of-care use [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] . These tests rely on serological markers 174 All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020 . . https://doi.org/10.1101 (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020. . All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted May 22, 2020 . . https://doi.org/10.1101 WHO Coronavirus Disease (COVID-19) Dashboard Estimating the asymptomatic proportion of coronavirus 203 disease 2019 (COVID-19) cases on board the Diamond Princess cruise ship Estimation of the asymptomatic ratio of novel coronavirus 206 infections (COVID-19) COVID-19 RT-PCR TEST Food and Drug Administration Serology-based tests for COVID-19 Developing a National Strategy 215 for Serology (Antibody Testing) in the United States. The Johns Hopkins Center 216 for Health Security COVID-19 Serology Surveillance Strategy Cellex qSARS-CoV-2 IgG/IgM Rapid Test Serological immunochromatographic approach in diagnosis with SARS-CoV-2 infected COVID-19 patients Performance of VivaDiag COVID-19 IgM/IgG Rapid Test is 227 inadequate for diagnosis of COVID-19 in acute patients referring to emergency 228 room department Comparison of Cepheid Xpert Xpress and Abbott ID Now 232 to Roche cobas for the Rapid Detection of SARS-CoV-2 Comparison of Abbott ID Now, Diasorin Simplexa, and CDC 235 FDA EUA methods for the detection of SARS-CoV-2 from nasopharyngeal and 236 nasal swabs from individuals diagnosed with COVID-19 Comparison of Abbott ID Now and Abbott m2000 methods 238 for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from 239 symptomatic patients Performance of the rapid Nucleic Acid Amplification by Abbott Advice on the use of point-of-care immunodiagnostic tests for COVID-19: 244 Scientific brief. World Health Organization