key: cord-0707147-zlmdvtto
authors: Trevor, R. A.; Grippon, S.; Chen, H.; Koh, L.; Borley, D.; Oladimeji, P.; Kidd, S.; Martin, J.
title: Meta-analysis of rapid direct-to-PCR assays for the qualitative detection of SARS-CoV-2
date: 2021-05-12
journal: nan
DOI: 10.1101/2021.05.07.21256745
sha: 7e50d87d3261bfcbfe002f6fb69a60e5e0f5231e
doc_id: 707147
cord_uid: zlmdvtto

Infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the ensuing COVID-19 pandemic present significant challenges to current diagnostic and therapeutic patient care pathways including whether new in vitro diagnostic tests can accurately identify and rule out current SARS-CoV-2 infection. The gold standard diagnostic test to identify a current SARS-CoV-2 infection is a central laboratory-based molecular assay using reverse transcription polymerase chain reaction (RT-PCR) with very high accuracy of detection but which typically requires 1-2 days turn-around for results. Therefore, rapid RTPCR assays and systems have been developed which can be deployed locally (near-patient or point of care (POC), provide faster results and not impact on already stressed central laboratory capacity. Rapid test results can be returned within the same clinical encounter, facilitating timely decisions that optimise the patient care pathway and support more rapid COVID-19 diagnosis, isolation and contract tracing activities. Direct-to-PCR is an evolution of RT-PCR in which the patient sample is added directly to an amplification reaction without being subjected to prior nucleic acid extraction, purification, or quantification to reduce the time and monetary resources required to process samples. Rapid, direct-to-PCR systems further increase the speed of testing by combining rapid PCR instruments with direct-to-PCR assays, to generate results in less than two hours. This appears to be the first meta-analysis assessing the accuracy of rapid direct-to-PCR in the detection of SARS-CoV-2. In total, 10,957 unique records were identified and screened using a search string evaluation, 420 full-text reports and/or supplemental materials were assessed for inclusion. This resulted in 14 studies reporting 20 datasets with 4593 patient samples (1391 positive) included in the analysis. The overall agreement between the rapid direct RT-PCR and gold standard centralised laboratory RT-PCR was 97.1% with 93.60% positive percent agreement and 98.63% negative percent agreement. The Cohens kappa statistical coefficient k = 0.93, indicating an almost perfect agreement and Youden Index = 0.92. These results indicate that direct-to-PCR assays can perform equivalently to the standard centralised laboratory PCR systems for the detection of SARS-CoV-2. Objectives: To assess the efficacy of rapid direct-to-PCR assays and systems for the detection of SARS-CoV-2 in the hospital, care home and medical research population in England from November 2020 to April 2021. Search methods: Electronic searches of the Cochrane COVID-19 Study Register (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) were undertaken on the 30th of April 2020. Please see the PRISMA flow diagram below (figure 2). Selection criteria: Studies of subjects with either suspected SARS-CoV-2 infection, known SARS-CoV-2 infection or known absence of infection, or those who were being screened for infection were included. Commercially available and research use rapid direct-to-PCR assays (without RNA extraction and purification reporting results within two hours) were included in the study. Data collection, extraction and analysis: Studies were screened independently, in duplicate with any disagreements resolved by discussion with a third author. Study characteristics were extracted by one author and checked by a second; extraction of study results and assessments of risk of bias and applicability were undertaken independently in duplicate. Where studies were not publicly available, sites that undertook in-service evaluations of rapid direct-to-PCR system were contacted and asked to supply anonymised datasets. Both reviewers independently performed data extraction and verification and calculated 2x2 contingency tables of the number of true positives, false positives, false negatives and true negatives. They resolved any disagreements by discussion and by review with the third reviewer. Main results: Twenty study cohorts were included (described in 14 study reports, including 4 unpublished reports), reporting results for 4593 samples (1391 with confirmed SARS-CoV-2). Studies were mainly from Europe and North America, and evaluated nine direct-to-PCR assays. Conclusions: This appears to be the first meta-analysis assessing the accuracy of rapid direct-to-PCR in the detection of SARS-CoV-2. In total, 10,957 unique records were identified and screened using a search string evaluation, 420 full-text reports and/or supplemental materials were assessed for inclusion. This resulted in 14 studies reporting 20 datasets with 4593 patient samples (1391 positive) included in the analysis. The overall agreement between the rapid direct RT-PCR and gold standard centralised laboratory RT-PCR was 97.10% with 93.60% positive percent agreement and 98.63% negative percent agreement. The Cohens kappa statistical coefficient k = 0.93, indicating an almost perfect agreement and Youden Index = 0.92. These results show that direct-to-PCR assay perform equivalently to the gold standard centralised laboratory RT-PCR systems for the detection of SARS-CoV-2.

What is a rapid direct-to-PCR tests for diagnosing COVID-19?

Rapid direct-to-PCR tests are rapid tests that aim to confirm or rule out the presence of SARS-CoV-2 within 2 hours without complicated processing of the sample.

We compared the accuracy of rapid direct-to-PCR tests with gold standard centralised laboratory RT-PCR for the detection of SARS-CoV-2 and found that direct-to-PCR was as accurate as standard RT-PCR assays.

People with suspected COVID-19 need to know quickly whether they are infected, so that they can self-isolate, inform close contacts and possibly receive treatment. Currently, COVID-19 infection is confirmed by a laboratory test called RT-PCR, which uses specialist equipment and often takes at least 24 hours to produce a result. If they . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021. ;  https://doi.org/10.1101/2021.05.07.21256745 doi: medRxiv preprint 4 4 are accurate, faster diagnosis could allow people to take appropriate action more rapidly, with the potential to reduce the spread of COVID-19. 1 What did we aim to find out?

Our goal was to determine if commercially available and research use rapid direct-to-PCR tests are accurate enough to detect SARS-CoV-2 compared to gold standard laboratory RT-PCR.

We looked for studies that measured the accuracy of any commercially produced and research use rapid directto-PCR tests, in people tested for COVID-19 using RT-PCR. People could be tested in hospital or in the community. Studies could test people with or without symptoms.

Tests had to use minimal equipment, be performed safely without risking infection from the sample, and have results available within two hours of the sample being collected.

We included 420 studies in the review. They investigated a total of 4593 nose or throat samples; COVID-19 was confirmed in 1391 of these samples. The studies investigated nine different direct-to-PCR tests. They took place mainly in Europe and North America.

Although overall results for diagnosing and ruling out COVID-19 were good (93% of infections correctly diagnosed and 99% correctly ruled out), we noted a difference in COVID-19 detection between tests, but we cannot be certain about whether results will remain the same in a real-world setting. We could not investigate differences in people with or without symptoms, nor time from symptom onset because the studies did not provide enough clinical information about their participants.

In general, the studies included followed rigorous methods in accordance with the tests intended use to detect COVID-19 and included at least two independent results to confirm or rule out COVID-19 infection. The results from different test brands varied and few studies compared multiple rapid-PCR tests. Most of the studies did not provide sufficient information to determine whether the detection levels would vary in people with COVID-19 symptoms versus without symptoms.

On average the rapid direct-to-PCR were shown to be equivalent to gold standard laboratory-based RT-PCR tests and several direct-to-PCR tests show very high accuracy. However, for most of the tests, more evidence is needed particularly in people without symptoms, on the accuracy of repeated testing, testing in non-healthcare settings such as schools (including self-testing), . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 12, 2021. ; https://doi.org/10.1101/2021.05.07.21256745 doi: medRxiv preprint 5 5

Infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the ensuing COVID-19 pandemic present significant challenges diagnostic and therapeutic challenges. These range from: understanding the value of signs and symptoms in predicting possible infection; assessing whether existing biochemical and imaging tests can identify infection or people needing critical care; and evaluating whether in vitro diagnostic tests can accurately identify and rule out current SARS-CoV-2 infection, and identify those with past infection, with or without immunity. 1 The standard diagnostic test to identify a SARS-CoV-2 infection is a central laboratory-based molecular assay using reverse transcription polymerase chain reaction (RT-PCR).

Direct-to-PCR is an evolution of PCR in which a sample is added directly to an amplification reaction without being subjected to prior nucleic acid extraction, purification, or quantification. It allows for maximum source quantities of RNA to be targeted, minimises opportunities for error and contamination, and may be less expensive, provide results more quickly and not require the same laboratory capacity, avoiding the need for centralised testing facilities.

Rapid direct-to-PCR systems further increase the speed of testing by combining rapid-PCR instruments with faster cycling times and data analysis with direct-to-PCR assays to generate results <2 hours.

If sufficiently accurate, rapid tests returned within the same clinical encounter can facilitate the timely decisions concerning the need for isolation and contract tracing activities 1 and to facilitate novel patient care pathways for the rapid diagnosis and treatment of COVID-19, and supporting infection prevention and control. 

Direct-to-PCR is an evolution of RT-PCR in which a sample is added directly to an amplification reaction without being subjected to prior nucleic acid extraction, purification, or quantification. It allows for maximum quantities of source RNA to be targeted, minimises opportunities for error and contamination, and reduces the time and monetary resources required to process samples. Rapid direct-to-PCR systems further increase the speed of testing by combining rapid-PCR instruments with faster cycling times and data analysis with direct-to-PCR assays, generating results in <2 hours.

The primary consideration for the eligibility of tests for inclusion in this review is that they were aimed at the detection of SARS-CoV-2 infection in samples without being subjected to prior nucleic acid extraction, purification, or quantification and with results in <2 hours.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021. ; https://doi.org/10.1101/2021.05.07.21256745 doi: medRxiv preprint

Clinical pathway

Patients may be tested for SARS-CoV-2 when they present with symptoms, have had known exposure to a confirmed case, or are part of a screening program, with no definite known exposure to SARS-CoV-2. The standard approach to diagnosis of SARS-CoV-2 infection is through laboratory-based testing of swab samples taken from the upper respiratory (e.g., nasopharynx, oropharynx) or lower respiratory tract (e.g., bronchoalveolar lavage or sputum) with RT-PCR. 1

It is essential to understand the clinical accuracy of tests and clinical features to identify the best way they can be used in different settings to develop effective diagnostic and management pathways for SARS-CoV-2 infection and disease. Estimates of accuracy from these reviews will help inform diagnosis, screening, isolation, and patientmanagement decisions. 1

To assess the efficacy of rapid direct-to-PCR systems for the qualitative diagnosis of SARS-CoV-2 in the hospital, care home and medical research population in England from November 2020 to April 2021.

The review was conducted following the Cochrane Rapid Review process and workflow (v.4) and we have referenced and followed the approach of the 2021 Cochrane Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection review 1 . We conducted the preliminary literature search for existing or ongoing systematic reviews using the Cochrane Library and estimated review feasibility. The protocol was drafted, and a team of systematic review authors screened the titles and abstracts of all records retrieved from the literature searches following the search string optimisation described in the appendix. Two review authors independently screened studies and a third, senior review author resolved any disagreements. We tagged all records selected and obtained the full texts for all studies flagged as potentially eligible. Two review authors independently screened the full texts, and we resolved any disagreements on study inclusion through discussion with a third review author.

The authors assessed and discussed the risk of bias and applicability concerns using the RoB 2.0 algorithm tailored to this review 2 , and concluded that the categorization 'some concerns' was appropriate (Figure 1 below) .

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021. ; https://doi.org/10.1101/2021.05.07.21256745 doi: medRxiv preprint 7 Figure 1 . A tool for assessing Risk Of Bias due to Missing Evidence in a synthesis (ROB-ME)

We analysed rapid direct-to-PCR assays and computed estimates by summing the counts of TP, FP, FN and TN across 2x2 tables and verifying the supplemental information in publications or source data for the unpublished studies. We made comparisons between analyses using the rapid direct-to-PCR assay and standard centralized lab assay. Any discrepant results were adjudicated using a 3rd reference assay.

We made no formal assessment of reporting bias but have indicated where we were aware that study results were available but unpublished.

The Minimum study duration was 1 month, and the target population encompassed symptomatic and asymptomatic residents and staff in care homes, patients in hospitals, healthcare workers and medical researchers.

Paired nasopharyngeal, oropharyngeal or nasal patient swabs were collected. One swab from each pair was analysed using the rapid direct-to-PCR system and the other swab was tested using the gold standard centralised . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) 

Three reviewers screened all titles and abstracts for eligibility and no non-English titles were reviewed.

Conducted with a pilot-tested form by two reviewer and verified by a third using excel to record the following:

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021. 

Authors were contacted if contingency table or source data was unavailable from the publication.

Published contingency tables were verified from results in the supplemental information and used to calculate overall (OPA), positive (PPA) and negative (NPA) percent agreement for published records. Source data was extracted and verified by two reviewers for unpublished studies to calculate contingency table and then calculate OPA, PPA and NPA as above.

Overall, percent positive and percent negative agreement, Cohen kappa (statistical coefficient to measure the agreement) and the Youden Index were calculated.

Results of the search 10, 957 unique records (published or preprints) were screened for inclusion. 420 records were selected for further assessment by review of the abstract and / or full-text reports and 103 studies for review of full-text reports and / or supplemental materials. Fourteen reports were selected for requiring assessment for inclusion in this review.

See Figure 2 for the PRISMA flow diagram of search and eligibility results.

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021. ; https://doi.org/10.1101/2021.05.07.21256745 doi: medRxiv preprint 11 

The overall agreement between the rapid direct PCR and standard centralised laboratory PCR was 97.10% with 93.60% positive percent agreement and 98.63% negative percent agreement and is summarized in Figure 3 . . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021. 

Heterogeneity between studies was examined by visually inspecting the forest plots of sensitivity and specificity (figure 4 ).

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021. ; https://doi.org/10.1101/2021.05.07.21256745 doi: medRxiv preprint

Rapid direct-to-PCR systems further aim to reduce the speed of testing by combining rapid-PCR instruments with direct-to-PCR assays, to generate results in <2 hours. If sufficiently accurate, rapid tests returned within the same clinical encounter can facilitate timely decisions concerning the need for isolation and contract tracing activities 1 and create novel patient care pathways for the rapid diagnosis and treatment of COVID-19,and support infection prevention and control.

[Background and here] This appears to be the first meta-analysis assessing the accuracy of rapid direct-to-PCR in the detection of SARS-CoV-2 and is the first iteration of this living review and will be updated in 3-6 months. In this meta-analysis 10,957 unique records were identified and screened using a search string evaluation, 420 full-text reports and/or supplemental materials were assessed for inclusion. This resulted in 14 studies reporting 20 data-sets with 4593 patient samples (1391 positive) included in the analysis.

Through this analysis, several potential sources of heterogeneity in the comparative evaluation methodology, whereby a new RT-PCR assay is compared with a reference RT-PCR assay were identified. Firstly, the direct-to-PCR reaction involves a dilution step whereas the more traditional RT-PCR involves a concentration step. In the dilution step, the quantity of nucleic acid in the sample is diluted potentially increases the likelihood of a false negative result. Conversely for the same viral load in an extracted PCR reaction, the concentration step concentrates or enriches the quantity of RNA potentially. Therefore, this could present a significant source of systematic bias. Furthermore, comparing assays with different gene targets adds another fundamental source of bias because detection of sub-genomic RNA may not be a suitable indicator of active replication/infection. Subgenomic was found to have accumulated and remain detectable 17 days after infection. 15 Therefore, the comparison of an assay that detects genomic RNA (ORF1ab) versus an assay detecting sub-genomic RNA (Egene) may introduce a further systematic bias.

Nevertheless, overall agreement between the rapid direct PCR and standard centralised laboratory PCR was 97.1% with 93.60% positive percent agreement and 98.63% negative percent agreement. The Cohen's kappa statistical coefficient k = 0.93, indicating an almost perfect agreement and Youden Index = 0.92. These results show that direct-to-PCR assays perform equivalently to the gold standard centralised laboratory PCR systems for the diagnosis of SARS-CoV-2.

This meta-analysis of the accuracy of rapid direct PCR for the diagnosis of SARS-CoV-2 included fourteen reports encompassing 21 datasets with a total of 3348 patient samples (1007 positive). The overall agreement between the rapid direct PCR and standard centralised laboratory PCR was 97.1% with 93.60% positive percent agreement and 98.63% negative percent agreement. The Cohen's kappa statistical coefficient k = 0.93, indicating an almost perfect agreement and Youden Index = 0.92. These results show that direct-to-PCR assays perform equivalently . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted May 12, 2021. ; https://doi.org/10.1101/2021.05.07.21256745 doi: medRxiv preprint to the standard centralised PCR systems for the diagnosis of SARS-CoV-2 and support the investigation of new rapid patient pathways for the detection and treatment of COVID-19

Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection

Tools for assessing risk of reporting biases in studies and syntheses of studies: a systematic review

An in-well direct lysis method for rapid detection of SARS-CoV-2 by real time RT-PCR in eSwab specimens

Head-to-head comparison of direct-input RT-PCR and RT-LAMP against RTqPCR on extracted RNA for rapid SARS-CoV-2 diagnostics. medRxiv

Multiplexed and extraction-free amplification for simplified SARS-CoV-2 RT-PCR tests. Analytical chemistry

Comparison of four molecular in vitro diagnostic assays for the detection of SARS-CoV-2 in nasopharyngeal specimens

Rapid and sensitive detection of SARS-CoV-2 RNA using the Simplexa™ COVID-19 direct assay

Extraction-free SARS-CoV-2 detection by rapid RT-qPCR universal for all primary respiratory materials

COVID-19 in vitro diagnostic near-patient testing in Care Environments using a cluster Randomised Open-label trial design in an East London care home population (CICERO) : DATA ON FILE

Frequency and duration of SARS-CoV-2 shedding in oral fluid samples assessed by a modified commercial rapid molecular assay

Evaluation of the diagnostic accuracy of a new point-of-care rapid test for SARS-CoV-2 virus detection

Practicability and analytical performances of the novel VitaPCRTM SARS-CoV-2 Assay during the second wave of COVID-19 epidemic in France

NHS in-service evaluation of exsig COVID-19 direct assay

Technical Validation of Primer Design Ltd PROmate™ COVID-19 Direct Workflow Solution IFU version 3

Covid-19 direct evaluation

SARS-CoV-2 genomic and subgenomic RNAs in diagnostic samples are not an indicator of active replication

International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted

Appendix1 outlines the Direct-to-PCR assays that were analysed in this review . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021. ; https://doi.org/10.1101/2021.05.07.21256745 doi: medRxiv preprint 18 18 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted May 12, 2021. ; https://doi.org/10.1101/2021.05.07.21256745 doi: medRxiv preprint