key: cord-0756095-bdanrjqn
authors: Baillargeon, R. H.; Seyer, X.; Bernard-Bedard, E.
title: Using rapid (point-of-care) tests for COVID-19: A decision analysis comparing the expected benefit of two screening strategies
date: 2021-04-17
journal: nan
DOI: 10.1101/2021.04.15.21255569
sha: 548800ba00444a85220fabb88c8617ace28ee012
doc_id: 756095
cord_uid: bdanrjqn

Background : Rapid tests for COVID-19 could be used to augment the otherwise limited laboratory-based testing capacity, but there are concerns that their utility may be compromised by their limited accuracy. The objective of this article is to compare the expected benefit (EB) of two screening strategies, one with rapid tests (SwRT) and another one without rapid tests (Sw RT). Methods : We performed a decision analysis, with the overall EB defined as the proportion of correctly identified individuals minus the proportion of incorrectly identified individuals. Accordingly, the SwRT strategy will be deemed a better screening strategy if its lesser EB for COVID-19 free individuals is more than compensated by its greater EB for COVID-19 individuals. Otherwise, it will not. Results : As expected, the EB for COVID-19 individuals was greater for the SwRT strategy, with a far superior ability to rule out the presence of COVID-19. In fact, under the scenario of interest (i.e., 8000 ID Now rapid tests in addition to 28185 lab-based RT-PCR tests), it identified almost 16% more COVID-19 individuals than the Sw RT strategy. In addition, the EB for COVID-19 free individuals was the same for both strategies, with a perfect ability at ruling in the presence of COVID-19. Conclusion : The SwRT strategy identified more COVID-19 individuals and this gain was not obtained at the detriment of COVID-19 free individuals who were equally well identified by both strategies. Hence, the SwRT strategy is a better screening strategy for COVID-19. It represents an opportunity to curtail the spread of SARS-CoV-2 that we may not afford to miss with new more contagious variants becoming more and more common in Canada.

Using rapid (point-of-care) tests for COVID-19: A decision analysis comparing the expected benefit of two screening strategies

The timely identification of COVID-19 individuals via testing is key to limit the spread of SRAS-CoV-2 in the community. 1 This may be especially true with new more contagious variants becoming more common in Canada, and the possibility of a 'third wave' looming at the horizon. Expert committees at the federal level 2 and in at least two provinces (i.e., Québec, Alberta) 3, 4 have considered the possibility of using rapid point-of-care (molecular and antigen) tests for COVID-19. But so far, efforts to implement rapid tests have fallen short as very few of the rapid tests being distributed (freely) to the provincial/territorial jurisdictions are being used.

For example, in Québec, Le Devoir was recently reporting than less than 2% are. 5 Rapid tests could be used to augment the otherwise limited lab-based testing capacity, but there are concerns that their utility may be compromised by their limited accuracy. If the goal is to have the most efficient screening strategy (see expected benefit benchmark below) then the question becomes whether the potential benefits of using rapid tests outweigh the costs? The objective of this article is to conduct a decision analysis to compare the overall expected benefit [EB; proportion of correctly identified individuals -proportion of incorrectly identified individuals] of two COVID-19 screening strategies, one with rapid tests (SwRT) and another one without rapid tests (S‫ݓ‬ ഥ RT). On the one hand, we predict the EB for COVID-19 individuals (i.e., proportion of true positive outcomes -proportion of false negative outcomes/not tested) to be higher for the SwRT strategy. On the other hand, we predict the EB for COVID-19 free individuals (i.e., proportion of true negative outcomes/not tested -proportion of false positive outcomes) to be smaller for the SwRT strategy. The SwRT strategy will be deemed a better screening strategy if its lesser EB for COVID-19 free individuals is more than compensated by its greater EB for COVID-19 individuals. Otherwise, it will not.

One way to compare the EB of the two screening strategies is to use decision analysis. 6 This formal process utilizes estimates from empirical studies to determine what the best strategy is. This information might include a test's sensitivity (SE) and specificity (SP) estimates obtained from a diagnostic test accuracy (DTA) study. Each strategy is depicted as a probability tree (diagram) that includes all possible events (e.g., obtaining a positive rapid test result for COVID-19 individuals). Events constitute the tree's branches. Mutually exclusive and exhaustive events appear at the right of chances nodes (represented by circles) together with their corresponding probabilities. Also, a probability tree includes all possible outcomes (e.g., being correctly identified as having COVID-19--true positive outcome). They appear at the far right of the probability tree. There is one for each possible sequence of events (i.e., paths). The (path) probability of a particular outcome is obtained by multiplying the probabilities associated with all the events leading to the outcome in question. (The probability of a particular event being conditional on all the events the precede it in the sequence.) Finally, the two strategies are linked by a decision node (represented by a square) and together they constitute a decision tree.

Empirical estimates such as a test's sensitivity (SE) and specificity (SP) often need to be calibrated (on a scale with values ranging from 0 to 1) before they can be used to assign probabilities to events in a probability tree. 7 An estimate of a test's SE depends on the proportion of individuals who obtained a positive test result in that particular study (i.e., the latter corresponding to the value of SE expected by chance alone). Also, an estimate of a test's SP depends on the proportion of individuals who obtained a negative test result in that particular All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 17, 2021. ; https://doi.org/10.1101/2021.04.15.21255569 doi: medRxiv preprint study (i.e., the latter corresponding to the value of SP expected by chance alone). Following Kraemer, 8 SE and SP can be calibrated using the weighted kappa coefficients k(1,0) and k(0,0), respectively. The coefficient k(1,0) is equal to (SE -SE expected by chance alone) / (1 -SE expected by chance alone). Similarly, k(0,0) is equal to (SP -SP expected by chance alone) / (1 -SP expected by chance alone). These calibrated values can then be used to obtain probabilities of events (e.g., obtaining a positive rapid test result for COVID-19 individuals) whatever the proportion of individuals with a positive/negative test result might be under the scenario of interest. Table 1 presents the information required to estimate the EB of the two screening strategies.

First, under the scenario of interest, the proportion of lab-based reverse transcription polymerase chain reaction (RT-PCR) tests was estimated at .779. For the reference period going from February 1 st to February 10 th , 281848 lab-based RT-PCR tests were performed at screening centers in Québec (i.e., an average of approximately 28185 tests per day) according to the Institut National de Santé Publique du Québec (INSPQ). 9 Also, from a recent press release from the Ministère de la Santé et des services sociaux (Québec's Health Ministry), 10 we postulated that, under the SwRT strategy, an additional 8000 Abbott ID Now tests could be performed on a daily basis. The ID Now is an isothermal nucleic acid amplification test that has been approved for commercial/diagnostic use by Health Canada (30 September 2020). 11 It is one of many rapid tests for COVID-19 that are being distributed (at no cost) by the Public Health Agency of Canada to the provincial/territorial jurisdictions. All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 17, 2021. ; https://doi.org/10.1101/2021.04.15.21255569 doi: medRxiv preprint Second, the relative prevalence of COVID-19 (at screening centers) was estimated at .0361 using data from the INSPQ. 9 In fact, for the mentioned above reference period, there were 10183 confirmed cases and 271665 non confirmed cases.

Third, the accuracy of lab-based RT-PCR tests for COVID-19 was assumed to be perfect.

Fourth, the calibrated values of the ID Now's SE and SP were estimated from a DTA study conducted by Moore and his colleagues. 12 According to a recent Cochrane DTA systematic review, 13 this is the only DTA study of the ID Now that used two lab-based RT-PCR tests. It is generally accepted that, to be considered valid for DTA purpose, a negative test result for COVID-19 needs to be confirmed by two lab-based RT-PCR tests. 13 We re-analyzed Moore et (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. For that reason, the two corresponding outcomes are labelled 'false negative' (not appropriate) and 'true positive' (appropriate), respectively. Table 2 presents, for each screening strategy, the averaged-out probabilities for the four outcomes of interest. First, as expected, the EB for COVID-19 individuals was greater for the SwRT (i.e., .0326 -.0035 = .029) than for the S‫ݓ‬ ഥ RT strategy (i.e., .0281 -.0080 = .020). In fact, the SwRT strategy' ability to rule out the presence of COVID-19 was far superior. Under the SwRT strategy, 90.2% of COVID-19 individuals were correctly identified. In comparison, only All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Overall, the EB benefit was greater for the SwRT (i.e., .029 + .964 = .993) than for the S‫ݓ‬ ഥ RT strategy (i.e., .020 + .964 = .984). The former identified more COVID-19 individuals and this gain was not obtained at the detriment of COVID-19 free individuals who were equally well identified by both strategies. Table 3 compares the EB of the two screening strategies under a broad range of values for the (unique) probabilities assigned to the different events that appear in the decision tree (see Figure 1 ). The evidence of favor of the SwRT strategy remains basically the same, except for a change in the number of rapid tests being administered. In fact, the difference between the two strategies in the proportion of COVID-19 individuals being correctly identified was estimated at only 7.89% if fewer (i.e., 4000 instead of 8000) rapid tests were administered--a gain of approximately 80 confirmed cases per day. In contrast, this differential was estimated at 23.67% if more (i.e., 12000 instead of 8000) rapid tests were administered--a gain of approximately 241 confirmed cases per day.

Beyond practical considerations, there are still many unanswered questions surrounding the use of readily available rapid (point-of-care) tests to augment the otherwise limited All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 17, 2021. ; https://doi.org/10.1101/2021.04.15.21255569 doi: medRxiv preprint laboratory-based testing capacity for COVID-19. The key question is whether the potential benefits of using rapid tests outweigh the costs. The goal of this article was to compare the EB of two screening strategies for COVID-19, one with rapid tests (SwRT) and another one without rapid tests (S‫ݓ‬ ഥ RT). The results show that the ability to rule out the presence of COVID-19 was greater under the SwRT, with 16% more COVID-19 individuals being correctly identified-approximately 161 confirmed cases per day for a province like Québec with a target of 35,000 tests per day. 14 Because the ability to rule in the presence of COVID-19 was the same for both strategies (i.e., 100%), this gain was not obtained at the detriment of COVID-19 free individuals.

Hence, on all accounts, the SwRT strategy is a better screening strategy for COVID-19.

Of course, if more rapid tests were administered, but also if a different, more sensitive, test was used instead of the ID Now. For instance, the rapid Cepheid Xpert Xpress molecular test has demonstrated a better clinical sensitivity in DTA studies. 13, 15 Also, it provides not only a qualitative result, but also a reading of the viral load of the patient (i.e., cycle threshold). This could greatly facilitate how we deal with negative rapid test results. For individuals with a low viral load (e.g., cycle threshold values above 30), 16 a negative test result could be systematically investigated using a lab-based RT-PCR test. But confirming or not confirming a negative test result may not mater much in terms of limiting the outbreak because contagiousness/transmission is most likely to occur when the viral load peaks. 17 In fact, in a recent simulation study, Larremore and his colleagues 18 have demonstrated that (under some circumstances) rapid tests may be as good as lab-based tests when it comes to stop the transmission of SARS-CoV-2. At any event, individuals with a negative rapid test result should continue to adhere to the usual All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted April 17, 2021. ; https://doi.org/10.1101/2021.04.15.21255569 doi: medRxiv preprint public health precautions (i.e., washing hands frequently, respecting social distancing, wearing masks and, if sick, staying at home and self isolate).

First, the accuracy of lab-based RT-PCR tests for COVID-19 was assumed to be perfect despite recent evidence that they may generate false-negative test results. [19] [20] [21] Had this been taken into account, the differential between the two strategies would only have been greater. Second, the above-cited Cochrane DTA systematic review 13 has since been updated, 22 but the more recent DTA studies of the ID Now do not seem to have used, as recommended, 13 two lab-based RT-PCR tests to confirm a negative test result. Hence, as far as we know, our reanalysis of the data from Moore et al. 12 represents the best attempt to date to obtain calibrated values of the ID Now's SE and SP. Third, this is a preprint version of this article, and therefore it has not been peer reviewed.

Experts who have cautioned against using rapid tests have done so based on their lack of perfect accuracy. Paradoxically, not using rapid tests amounts to give a negative test results to all these individuals (including those with COVID-19) who could have been tested otherwise (had rapid tests been implemented). Not testing might be a way to avoid false-positive test results, but it is not an issue here, with the specificity of rapid tests being comparable to the one of lab-based RT-PCR tests. Thus, a screening strategy using (valid) rapid tests to augment our testing capacity can allow us to identify more COVID-19 individuals. Moreover, it does not do so at the detriment of COVID-19 free individuals. It is a net gain. It represents an opportunity that we may not afford to miss at a time where new more contagious variants are becoming more common in Canada. What are we waiting for? All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. Authors' contributions: RHB conceived the study. All of the authors were involved in the data analysis. RHB wrote the first draft of the article and all the authors contributed to its critical revision. All of the authors approved the final version of the article and are accountable for all of its aspects.

The authors did not receive funding for this work. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. ഥ RT strategy, respectively. Because k(0,0) / k(1,0) = odds ratio for the marginal probabilities, the sensitivity results for SE and the marginal odds are mirror images of each other. SwRT: screening with rapid tests. S‫ݓ‬ ഥ RT: screening without rapid tests.

All rights reserved. No reuse allowed without permission.

(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. 

The implications of silent transmission for the control of COVID-19 outbreaks

Priority strategies to optimize testing and screening for COVID-19 in Canada

Avis sur l'utilisation des tests rapides au point de service pour la détection de la COVID-19 au Québec

Scientific Advisory Group Rapid Evidence Report

Croulant sous les ventilateurs inutilisés, Ottawa essaie d'annuler les contrats restants

Cambridge All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity

Clinical practice guidelines for monitoring children's behavioural development at the 18-month well-baby visit: A decision analysis comparing the expected benefit of two alternative strategies

Evaluating medical tests: Objective and quantitative guidelines

Québec accélère l'usage des tests rapides ID NOW

Comparison of Two Commercial Molecular Tests and a Laboratory-Developed Modification of the CDC 2019-nCoV Reverse Transcriptase PCR Assay for the Detection of SARS-CoV-2

Cochrane Database of Systematic Reviews. 2020(8):133. 14. Government of Canada. Safe Restart Agreement. Available at: All rights reserved. No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. The copyright holder for this preprint this version posted

The diagnostic accuracy of isothermal nucleic acid point-of-care tests for human coronaviruses: A systematic review and metaanalysis

Comparison of Cepheid Xpert Xpress and Abbott ID Now to Roche cobas for the Rapid Detection of SARS-CoV-2

Temporal dynamics in viral shedding and transmissibility of COVID-19

Test sensitivity is secondary to frequency and turnaround time for COVID-19 screening

False-negative results of initial RT-PCR assays for COVID-19: A systematic review

Clinical sensitivity and interpretation of PCR and serological COVID-19 diagnostics for patients presenting to the hospital

Rapid, point of care antigen and molecular based tests for diagnosis of SARS CoV 2 infection

No reuse allowed without permission. (which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity

We would like to thank Nigèle Langlois for her help with searching the literature. Also, we would like to thank Marc Bélanger for his help with the figure. Finally, we would like to thank all with whom we exchanged correspondence while writing this paper.Correspondence to: Raymond H. Baillargeon, raymond.baillargeon@uottawa.ca