key: cord-0771857-ha3vh13c
authors: Böger, Beatriz; Fachi, Mariana M.; Vilhena, Raquel O.; Cobre, Alexandre de Fátima; Tonin, Fernanda S.; Pontarolo, Roberto
title: Systematic review with meta-analysis of the accuracy of diagnostic tests for COVID-19
date: 2020-07-10
journal: Am J Infect Control
DOI: 10.1016/j.ajic.2020.07.011
sha: bd7011c20536016b8b3a959bab481cd79e34428a
doc_id: 771857
cord_uid: ha3vh13c

OBJECTIVE: . To collate the evidence on the accuracy parameters of all available diagnostic methods for detecting SARS-CoV-2. METHODS: . A systematic review with meta-analysis was performed. Searches were conducted in Pubmed and Scopus (April 2020). Studies reporting data on sensitivity or specificity of diagnostic tests for COVID-19 using any human biological sample were included. RESULTS: . Sixteen studies were evaluated. Meta-analysis showed that computed tomography has high sensitivity (91.9% [89.8–93.7%]), but low specificity (25.1% [21.0–29.5%]). The combination of IgM and IgG antibodies demonstrated promising results for both parameters (84.5% [82.2%-86.6%]; 91.6% [86.0%-95.4%], respectively). For RT-PCR tests, rectal stools/swab, urine, and plasma were less sensitive while sputum (97.2% [90.3–99.7%]) presented higher sensitivity for detecting the virus. CONCLUSIONS: . RT-PCR remains the gold standard for the diagnosis of COVID-19 in sputum samples. However, the combination of different diagnostic tests is highly recommended to achieve adequate sensitivity and specificity.

Patients report fever and cough, and most develop chest discomfort, difficulty in breathing or pneumonia, being clinically diagnosed by imaging tests such as chest X-ray or computed tomography (CT). CT equipment is widespread worldwide and the scan process is relatively simples and quick, which enables rapid screening for suspected patients. The typical findings of chest CT images for individuals with COVID-19 are multifocal bilateral patchy ground-glass opacities or consolidation with interlobular septal and vascular thickening in the peripheral areas of the lungs. However, CT findings can change as the disease progresses and these manifestations may also be compatible with other viral pneumonias 4,5 . In this context, the current gold standard for diagnosing COVID-19 is based on a molecular test of the reverse transcription polymerase chain reaction (RT-PCR), aimed at detecting the RNA of the virus in respiratory samples such as nasopharyngeal swabs or bronchial aspirate 6 . The real-time RT-PCR test provides a sensitive (the ability of the test to correctly identify those patients with the disease 7, 8 ) and specific (the ability of the test to correctly identify those patients without the disease 8 ) method to detect SARS-COV-2, with different diagnosis protocols including sequences of target primers available in the World Health Organization public database 6, 9 . However, researchers should be aware that this test can also give false negatives if the amount of viral genoma is insufficient or if the correct time-window of viral replication is missed 10 . Although the COVID-19 incubation period is estimated to be 5 days, false negative results are common within 7 days of infection. Additionally, RT-PCR process is time-consuming and shortages in test kit supplies are common worldwideespecially during the beginning of the epidemic outbreak 11 .

Other simpler and rapid methods, such as serological testing of IgM and IgG production in response to viral infection, can be used to enhance the detection sensitivity and accuracy of the molecular test or for screening purposes to assess antibody profiles in a large population 12, 13 . Because antibodies are usually detected only 1-3 weeks after the onset of symptoms, these tests are used to assess the overall infection rate in the communityincluding the rate of asymptomatic infectionsor in remote areas where qPCR assays are not available 12, 14 .

In this scenario, given the limitations of clinical diagnosis alone (due to the similarity of the symptoms of COVID-19 infection with those of other viruses) and the availability of different molecular and serological tests with both technical advantages and disadvantages, it is important to summarize the accuracy parameters of these methods and investigate whether they are sufficiently specific or sensitive to fit their role in practice. Few studies addressing the diagnostic performance of tests for COVID-19 exist, with special focus only on commercially assays available in a given country 42 Thus, we aimed to perform a systematic review with meta-analysis to gather evidence on the features of all available diagnostic test for SARS-CoV-2, including parameters of sensitivity, specificity, positive and negative likelihood ratios and summary receiver operating characteristic (SROC) curves, whenever possible.

Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) statement and Cochrane Collaboration recommendations 15, 16 .

Systematic searches were conducted in Pubmed and Scopus without limits of time-frame or language (last updated April 2020). The search strategy included the following descriptors: "diagnostic", "test", "assay", "covid-19", "sars-cov-2"and other terms combined with Boolean operators AND and OR.

The complete strategy is available in the supplementary material. Manual searches in the references lists of included studies and in the gray literature (e.g. Google Scholar) were also performed.

Titles and abstracts of retrieved articles were screened for eligibility. Relevant articles were read in full and those fulfilling inclusion criteria had their data extracted. Two authors performed all the literature selection steps individually and then discussed the differences with a third author.

Studies were included in this systematic review if they met all the following eligibility criteria:

(i) evaluation of any diagnostic method; (ii) aimed at diagnosis of SARS-CoV-2 (COVID-19); (iii) using any human biological sample; (iv) reporting data on the accuracy of the test (e.g. sensitivity and/or specificity). We excluded studies published in non-Roman characters.

The following data were independently extracted by two researchers: general study details (authors, year of publication, country of origin, study design, sample size), methods, characteristics, and diagnostic test results (true positive, TP; true negative, TN; false positive, FP; false negative, FN, sensitivity, specificity, accuracy).

Two reviewers evaluated independently the risk of bias in each study using the Diagnostic Precision Study Quality Assessment Tool (QUADAS-2) recommended by the Cochrane Collaboration.

The assessment was performed using the Review Manager Software version 5.3 17

The meta-analyses were performed according to the technique and type of sample from each study (i.e. by subgroups). Sensitivity, specificity, positive likelihood ratio (PLR) and negative likelihood ratio (NLR) were measured with a 95% confidence interval based on the TP, TN, FP and FN rates that were extracted from the included studies.

Sensitivity, defined as the probability that a test result will be positive when the disease exists (true positive rate) was calculated as = VP/(VP+FN). Specificity, defined as the probability that a test result will be negative when the disease is not present (true negative rate) was calculated as = VN/(VN+VP). The PLR is the ratio between the probability of a positive test result given the presence of the disease and the probability of a positive test result given the absence of the disease, i.e. = true positive rate/false positive rate, or expressed as sensitivity/(1-specificity). The NLR is ratio between the probability of a negative test result given the presence of the disease and the probability of a negative test result given the absence of the disease, i.e. = false negative rate/true negative rate, or expressed as (1sensitivity)/specificity. Summary receiver operating characteristic (SROC) curves based on TP e FP rates were also built whenever possible to describe the relationship between test sensitivity and specificity. An area under the curve (AUC) close to 1 indicated a good diagnostic performance of the test. All analyses were performed using the Meta-Disc© version 1.4.7.

The heterogeneity of the studies was established by χ 2 analysis, with inconsistency values (I 2 )

greater than 50% being considered as moderate heterogeneity, and I 2 greater than 75% defined as high heterogeneity. Outcomes with I 2 values greater than 50% were submitted to sensitivity analysis (i.e., hypothetical removal of studies).

A total of 1089 articles were identified after duplicate removal. Of these, 1046 were excluded during the screening phase (title and abstract reading), with 43 records being fully appraised. Sixteen studies were included finally in the systematic review [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] . We were able to include 14 trials in the quantitative analyses (meta-analysis): the studies by Corman et al. 22 and Pfefferle et al. 33 All studies included in this review (n = 2297 patients) were published in 2020, designed as retrospective observational cohorts, with only one defined as a control-case study 32 20, 27, 28, 30 , and stool and rectal swabs 20, 27, 28, 31 . Table 1 summarizes the main characteristics of the included studies. Table 2 ). The genes E and RdRp were the most commonly used to detect the COVID-19 virus, both with high analytical sensitivity (technical limit of detection of 3.2 and 3.6 copies per reaction, respectively).

The detection of the gene N presented lower analytical sensitivity ( 

Meta-analyses evaluating the parameters of accuracy (sensitivity, specificity, PLR and NLR) of the reported tests were performed (Table S1) , results are shown in Table 3 . Figure S1 of the supplementary material for complete results).

Immunological tests (IgM and IgG) were evaluated in five trials as a diagnostic method for COVID-19 21, 24, 25, 27, 34 . The antibody dosage was tested in whole blood samples 21, 32 , fingerstick blood 24 , serum 24, 25, 27 , and plasma 24 . Overall, sensitivity and specificity were higher when the combination of IgM and IgG antibodies was evaluated (see Figure S2 , S3 and S4), reaching 84.5% (95% CI 82.2-86.6%, I 2 = 93.2%) and 91.6% (95% CI 86.0-95.4%, I 2 = 0%) respectively. The SROC curves for the immunological diagnostic tests are shown in Figure 2 . analyses were performed for all meta-analyses with high heterogeneity results (I 2 > 50%); however, no additional differences were found compared to the effects of the original analyses (data not shown).

Studies were rated as being of moderate overall methodological quality according to QUADAS-2 (see Figures 3 and 4) . The studies by Chan et al. 20 and Pfefferle et al. (29) were not evaluated given the lack of clinical application of the tests (i.e., only the analytical performance of the methods was assessed). Around one-quarter of the trials (26%) did not describe the methods of patient selection, and almost half (46%) included previously diagnosed patients, which may enhance the risk of bias. However, the majority of the patients included matched the review question and were likely to be diagnosed with the evaluated tests (i.e., no major concerns for the applicability domain). Overall, 80% of the studies properly reported both index and reference standard tests and how they were conducted and interpreted.

Only three studies (20%) properly reported the interval between tests, whether patients received different index or standard assays, and the complete statistical analyses performed, thus being judged as having low risk of bias for the flow and timing domain. The remaining studies were classified as with an unclear risk of bias for this domain.

To our knowledge, this is the first systematic review with meta-analysis to collate the available evidence on the accuracy parameters of different diagnostic methods (clinical, molecular, and serological)

for the detection of SARS-CoV-2 in different samples, including blood, nasopharyngeal swab, sputum, saliva, urine and feces. We were also able to evaluate qualitatively the main analytical parameters reported in the molecular techniques. 35, 36 . In the past months, different RT-PCR kits for the detection of SARS-CoV-2 have been developed, being able to amplify a small amount of viral genetic material in a sample 37 . In this technique, the RNA of the virus is reverse-transcribed into complementary DNA strands (cDNA), whose specific regions are amplified. The process usually involves two main steps: sequence alignment and primer design, and assay optimization and testing, especially because this method requires several temperature changes for each cycle using thermocycling equipment 35 .

We 42 , indicating a high level of accuracy for immunological tests in the diagnosis of COVID-19. The antibodies researched in these tests refer to structural antigenic proteins of SARS-CoV-2, such as spike (S) and nucleocapsid (N) proteins, which have been identified as the most relevant in the development of serological assays for the diagnosis of the infection 14 . Usually, the body's immune response to a pathogen takes 1-2 weeks to occur. In this context, the use of serological tests for detection in the initial/acute phase of the disease can be challenging. A recent study showed that IgM and IgG seroconversion can occur simultaneously or sequentially in COVID-19, and that antibody titers reach a plateau after 6 days 19 . In addition, a meta-analysis of the accuracy of diagnostic tests marketed in Brazil, taken from manufacturers' data, showed a range of 10-40% false-negative results for detection of SARS-CoV-2 IgM in the acute phase in eight evaluated tests 42 . However, immunological tests have a quick turnaround time and relatively low costs (around £6 per test or USD$ 8-10) 43 , which may represent an important strength for this method, given the shortages of RT-PCR and its higher price (around £30/test, but may range from USD$ 25 to 100). Additionally, the participation of multiple manufacturers in the market can potentially scale the immunological tests to millions of people per day due to their simpler design. This may especially help to improve the detection of the virus in healthcare settings where resources are more limited, such as in developing countries 43, 44 .

Our study has some limitations. The included studies differ in terms of size, risk of bias, and external validity. We are aware of potential introduction of bias caused by studies of poor methodological quality. We found high heterogeneity rates among trials, probably cause by some differences in the methods, patient characteristics, and samples used. However, we tried to avoid systematic errors by performing sensitivity analyses, which do not demonstrate significant differences from the original analyses. The eligibility criteria and description of the participants is crucial, as the test is only valid under similar circumstances. Sensitivities, specificities, TP, and TN were compared, but these statistics depend on the populations studied, the reference tests used, and the specific function of the test. Studies were judged as being of moderate methodological quality, with few concerns regarding the applicability of the methods used. The low reporting quality of some studies hampered the performance of further analyses.

COVID-2019) situation reports

Current Status of Epidemiology, Diagnosis, Therapeutics, and Vaccines for Novel Coronavirus Disease 2019 (COVID-19)

Epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (COVID-19) during the early outbreak period: a scoping review

A Novel Coronavirus from Patients with Pneumonia in China

Chest CT manifestations of new coronavirus disease 2019 (COVID-19): a pictorial review

COVID-19) technical guidance:Laboratory testing for 2019-nCoV in humans

Group Testing for COVID-19: How to Stop Worrying and Test More

Clinical tests: sensitivity and specificity. Continuing Education in Anaesthesia Critical Care & Pain

Combination of RT-qPCR testing and clinical features for diagnosis of COVID-19 facilitates management of SARS-CoV-2 outbreak

SARS-CoV-2 Viral Load in Upper Respiratory Specimens of Infected Patients. The New England journal of medicine

The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application

Profiling Early Humoral Response to Diagnose Novel Coronavirus Disease (COVID-19)

Serological assays for emerging coronaviruses: challenges and pitfalls

The Laboratory Diagnosis of COVID-19 Infection: Current Issues and Challenges

Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement

QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies

Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases

Diagnosis of the Coronavirus disease (COVID-19): rRT-PCR or CT?

Improved molecular diagnosis of COVID-19 by the novel, highly sensitive and specific COVID-19-RdRp/Hel real-time reverse transcription-polymerase chain reaction assay validated in vitro and with clinical specimens

Performance of VivaDiagTM COVID-19 IgM/IgG Rapid Test is inadequate for diagnosis of COVID-19 in acute patients referring to emergency room department

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro surveillance : bulletin Europeen sur les maladies transmissibles

CT image visual quantitative evaluation and clinical classification of coronavirus disease (COVID-19)

Development and Clinical Application of A Rapid IgM-IgG Combined Antibody Test for SARS-CoV-2 Infection Diagnosis

Evaluation of Nucleocapsid and Spike Protein-based ELISAs for detecting antibodies against SARS-CoV-2

Potential false-negative nucleic acid testing results for Severe Acute Respiratory Syndrome Coronavirus 2 from thermal inactivation of samples with low viral loads

Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study. The Lancet. Infectious diseases

Comparison of different samples for 2019 novel coronavirus detection by nucleic acid amplification tests

Imaging and clinical features of patients with 2019 novel coronavirus SARS-CoV-2

Quantitative Detection and Viral Load Analysis of SARS-CoV-2 in Infected Patients

Fecal specimen diagnosis 2019 novel coronavirus-infected pneumonia

Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease 2019. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America

Evaluation of a quantitative RT-PCR assay for the detection of the emerging coronavirus SARS-CoV-2 using a high throughput system. Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin

Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease 2019. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America

Diagnosing COVID-19: The Disease and Tools for Detection

Genome Composition and Divergence of the Novel Coronavirus (2019-nCoV) Originating in China

Assay Techniques and Test Development for COVID-19 Diagnosis

Novel Coronavirus (2019-nCoV) -Real-Time RT-PCR Diagnostic Panel -Catalog # 2019-nCoVEUA-01 1000 reactions

Interim Guidelines for Collecting, Handling, and Testing Clinical Specimens from Persons for Coronavirus Disease 2019 (COVID-19)

Prolonged viral shedding in feces of pediatric patients with coronavirus disease 2019

Chinese experience and recommendations concerning detection, staging and follow-up

COVID-19: a metaanalysis of diagnostic test accuracy of commercial assays registered in Brazil. The Brazilian Journal of Infectious Diseases

Unique epidemiological and clinical features of the emerging 2019 novel coronavirus pneumonia (COVID-19) implicate special control measures

COVID-19) Update: FDA Authorizes First Antigen Test to Help in the Rapid Detection of the Virus that Causes COVID-19 in Patients

The authors express their gratitude for research funding to the CAPES (Brazilian Federal Agency for Support and Evaluation of Graduate Education within the Ministry of Education of Brazil) -Finance Code 001. The authors would like to thank the Secretariat of Science, Technology and Higher Education (SETI-PR) for the financial support for laboratory infrastructure.

The authors have declared no conflict of interest.

RT-PCR remains the gold standard for the diagnosis of COVID-19 in sputum samples. However, depending on the type of sample and stage of the disease, other methods are preferable. A combination of clinical, molecular, and serological diagnostic tests is highly recommended to achieve adequate sensitivity and specificity. Automated assays for molecular diagnosis using a two-target system for detecting SARS-CoV-2 should be used whenever possible to enhance analytical performance.