key: cord-0696644-4g3xbll3 authors: Zhang, John J. Y.; Lee, Keng Siang; Ong, Chee Wui; Chan, Mae Yee; Ang, Li Wei; Leo, Yee Sin; Chen, Mark I‐Cheng; Lye, David Chien Boon; Young, Barnaby Edward title: Diagnostic performance of COVID‐19 serological assays during early infection: A systematic review and meta‐analysis of 11 516 samples date: 2021-02-20 journal: Influenza Other Respir Viruses DOI: 10.1111/irv.12841 sha: a5ae2cf5a9e970302d9fbec42c2e73dc750a1b69 doc_id: 696644 cord_uid: 4g3xbll3 OBJECTIVE: The use of coronavirus disease 2019 (COVID‐19) serological testing to diagnose acute infection or determine population seroprevalence relies on understanding assay accuracy during early infection. We aimed to evaluate the diagnostic performance of serological testing in COVID‐19 by providing summary sensitivity and specificity estimates with time from symptom onset. METHODS: A systematic search of Ovid MEDLINE, Embase, Cochrane Central Register of Controlled Trials (CENTRAL) and PubMed was performed up to May 13, 2020. All English language, original peer‐reviewed publications reporting the diagnostic performance of serological testing vis‐à‐vis virologically confirmed SARS‐CoV‐2 infection were included. RESULTS: Our search yielded 599 unique publications. A total of 39 publications reporting 11 516 samples from 8872 human participants met eligibility criteria for inclusion in our study. Pooled percentages of IgM and IgG seroconversion by Day 7, 14, 21, 28 and after Day 28 were 37.5%, 73.3%, 81.3%, 72.3% and 73.3%, and 35.4%, 80.6%, 93.3%, 84.4% and 98.9%, respectively. By Day 21, summary estimate of IgM sensitivity was 0.872 (95% CI: 0.784‐0.928) and specificity 0.973 (95% CI: 0.938‐0.988), while IgG sensitivity was 0.913 (95% CI: 0.823‐0.959) and specificity 0.960 (95% CI: 0.919‐0.980). On meta‐regression, IgM and IgG test accuracy was significantly higher at Day 14 using enzyme‐linked immunosorbent assay (ELISA) compared to other methods. CONCLUSIONS: Serological assays offer imperfect sensitivity for the diagnosis of acute SARS‐CoV‐2 infection. Estimates of population seroprevalence during or shortly after an outbreak will need to adjust for the delay between infection, symptom onset and seroconversion. On March 12, 2020, the World Health Organisation (WHO) declared the outbreak of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and associated coronavirus disease 2019 (COVID- 19) as a pandemic. Unprecedented control measures have been put in place in an effort to reduce transmission, which at their peak is estimated to have covered a third of the global population. [1] [2] [3] Despite these measures, by July 15, 2020, 13.5 million confirmed cases had been reported worldwide with 581 221 deaths-a crude case fatality rate of 4.3%. 4 However, the true number of infections (and deaths) is likely to be substantially higher due to the large proportion of infections which are undiagnosed because of atypical, mild or absent symptoms, or unconfirmed because testing was not available. [5] [6] [7] [8] Serological assays have the potential to play an important role in the surveillance of COVID-19. The results of early seroprevalence studies have indicated that during the first wave of the COVID-19 pandemic, up to 10% of the population of Wuhan, China may have been infected, and up to 33% in cities of other countries have experienced large outbreaks. 9, 10 Serology can also be used for the diagnosis of acute infection and can form an important tool in containment strategies by identifying and linking clusters of infection. 11 If a serological immune correlate of protection can be found, these assays may also form part of exit strategies from control measures. 12 While serological assays have been reported to have high sensitivity and specificity for SARS-CoV-2 infection, this reflects diagnostic performance during convalescence. 13 Understanding antibody kinetics during early SARS-CoV-2 infection is critical for assessing the accuracy of diagnostic serological results and for interpreting the results of seroprevalence studies. To address this issue, we conducted a systematic review and meta-analysis to evaluate the diagnostic performance of serological assays in early COVID-19, when compared to polymerase chain reaction (PCR) as the gold standard. This review was conducted in accordance to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. 14 A search string comprising synonyms of "COVID-19" and "serological assays" was applied to the following databases: Ovid MEDLINE, Embase, Cochrane Central Register of Controlled Trials (CENTRAL) and PubMed from November 1, 2019 to May 13, 2020 (Table S1 ). Studies were screened independently by two reviewers (JJYZ and KSL) with disagreements resolved by consensus or appeal to a third senior reviewer (BEY). Agreement between the reviewers on study inclusion was evaluated using Cohen's κ. 15 All English language, original peer-reviewed publications reporting the diagnostic performance of serological testing in comparison with virologically confirmed SARS-CoV-2 infection were included. Specific inclusion and exclusion criteria are outlined in Table S2 . The quality of included studies was assessed using QUADAS-2 (Table S3 and Figure S1 ). 16 In summary, the QUADAS-2 tool consists of four key domains that discuss patient selection, index test, reference standard and flow of patients through the study and timing of the index tests and reference standard. Two researchers (KSL and CWO) assessed the quality of all included studies and discussed discrepancies until consensus was reached. Data were extracted on the following variables: study, sample and patient details, method of diagnosis, type of blood sample and immunoassay, commercial name of test kit, cut-off values Random effects models were used for meta-analyses of variables and end points. 17 Pooled proportions were computed with the inverse variance method using the variance-stabilizing Freeman-Tukey double arcsine transformation. 18 Confidence intervals (CI) for individual studies were calculated using the Wilson Score confidence interval method with continuity correction. The I 2 statistic was used to present between-study heterogeneity, where I 2 ≤ 30%, between 30% and 50%, between 50% and 75%, and ≥75% were considered to indicate low, moderate, substantial and considerable heterogeneity, respectively. 19 P values for the I 2 statistic were derived from the chi-square distribution of Cochran Q test. For pooling of means of numerical variables, we computed missing means and standard deviations (SDs) from medians, ranges and interquartile ranges using the methods proposed by Hozo with additional likelihood-ratio tests was performed to evaluate for any significant effects of covariates. In addition, univariate meta-analysis was also done using random effect estimation with the DerSimonian-Laird method to produce pooled diagnostic odds ratios. 24 Publication bias was assessed using funnel plots and Egger's regression test, based on a weighted linear regression of the treatment effect on its standard error. 25, 26 All statistical analyses were performed using R software version 3.4.3 , with the packages meta and mada. 27, 28 P values less than .05 were considered statistically significant. Table S4 . The type of immunoassay used was specified in 47 studies. Among the 39 publications, the proportion of studies with low, high and unclear risk of bias and concerns regarding applicability are summarized in Table S3 and Figure S1 . More than half of the studies were found to be at high risk of bias under the domains "Patient Selection and Index Test." The main causes for high risk of bias were due to non-cohort study designs and cut-off values not being reported for the respective serological assays. Of the 11 516 samples analysed in total, 5743 were taken from laboratory-diagnosed COVID-19 patients, 5265 were from healthy controls and 508 were from patients infected with human coronaviruses (229E, HKU1, NL63, OC43 and SARS-CoV (using convalescent samples)), influenza A and B, and other respiratory pathogens. (Table 1 and Figure 1 ). No evidence of publication bias for IgM seroconversion was identified on Egger's regression test (P = .69; Figure S4 ). Subgroup meta-analysis identified no significant difference in IgM seroconversion rates between the various types of immunoassay (P = .44; Figure S5 ). Figure 1 ). No evidence of publication bias for IgG seroconversion was identified on Egger's regression test (P = .70; Figure S4 ). Subgroup meta-analysis identified no significant difference in IgG seroconversion rates between the various types of immunoassay (P = .56; Figure S7 ). Table S5 . On meta-regression, ELISA was found to have higher specificity values than CLIA (P = .011), with similar sensitivity values (P = .67). The likelihood-ratio test further suggested a significant difference in test accuracy values with CLIA versus ELISA (χ 2 = 6.62, P = .036). ELISA also demonstrated higher specificity values than ICA (P = .021), with comparable sensitivity values (P = .51). On further comparison using the likelihood-ratio test, however, the difference in diagnostic accuracy between the two tests was only close to statistical significance (P = .072). Our meta-analysis indicated that the overall sensitivity and specific- while cross-reactivity with other human coronaviruses was not evident from the diagnostic performance of assays included in our review, this may become an issue with less specific but more sensitive testing. An additional question is how these tests will perform in the event of further waves of COVID-19 where a substantial proportion of the population may already have been infected. IgM can persist for months, reducing the specificity of serological diagnosis. 69 Our study demonstrated that the utility of serological assays is limited to IgG more than 14-21 days after symptom onset, when most infected individuals have seroconverted and high sensitivity and specificity values are attained. Diagnostic testing is hence likely to continue to require virological confirmation such as PCR, while seroprevalence studies will significantly underestimate the proportion of a population infected if conducted too early in the epidemic curve. Longer-term studies would be beneficial to improve our understanding of the diagnostic performance of serological testing for COVID-19 in other epidemiological contexts such as endemic infection or future epidemics. The authors declare no competing interests. The peer review history for this article is available at https://publo ns.com/publo n/10.1111/irv.12841. The data that support the findings of this study are available in the Association of public health interventions with the epidemiology of the COVID-19 outbreak in Wuhan, China Responding to the COVID-19 outbreak in Singapore: staff protection and staff temperature and sickness surveillance systems What share of the world population is already on COVID-19 lockdown? Statista Coronavirus disease (COVID-2019) situation reports Presumed asymptomatic carrier transmission of COVID-19 Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia Interrupting transmission of COVID-19: lessons from containment efforts in Singapore Risk factors of severe disease and efficacy of treatment in patients infected with COVID-19: a systematic review, meta-analysis and meta-regression analysis The infection fatality rate of COVID-19 inferred from seroprevalence data Serological tests facilitate identification of asymptomatic SARS-CoV-2 infection in Wuhan Investigation of three clusters of COVID-19 in Singapore: implications for surveillance and response measures What policy makers need to know about COVID-19 protective immunity Connecting clusters of COVID-19: an epidemiological and serological investigation Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement A coefficient of agreement for nominal scales QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies A basic introduction to fixed-effect and random-effects models for meta-analysis Metaprop: a Stata command to perform meta-analysis of binomial data Cochrane handbook for systematic reviews of interventions version 5.1.0. The Cochrane Collaboration Estimating the mean and variance from the median, range, and the size of a sample Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range Confidence intervals for predictive values with an emphasis to case-control studies Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews Meta-analysis in clinical trials Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials Bias in meta-analysis detected by a simple, graphical test meta: an R package for meta-analysis. R News Meta-analysis of diagnostic accuracy with mada Performance characteristics of the Abbott architect SARS-CoV-2 IgG assay and seroprevalence in A peptide-based magnetic chemiluminescence enzyme immunoassay for serological diagnosis of coronavirus disease 2019 (COVID-19) Performance of VivaDiag COVID-19 IgM/IgG Rapid Test is inadequate for diagnosis of COVID-19 in acute patients referring to emergency room department Rapid and sensitive detection of anti-SARS-CoV-2 IgG, using lanthanide-doped nanoparticles-based lateral flow immunoassay Dynamic profile for the detection of anti-SARS-CoV-2 antibodies using four immunochromatographic assays Rapid point-of-care testing for SARS-CoV-2 in a community screening setting shows low sensitivity Detection of antibodies against SARS-CoV-2 in patients with COVID-19 Letter to the Editor: Three cases of re-detectable positive SARS-CoV-2 RNA in recovered COVID-19 patients with antibodies Detection of serum immunoglobulin M and immunoglobulin G antibodies in 2019-novel coronavirus infected cases from different stages Profiling early humoral response to diagnose novel coronavirus disease (COVID-19) Evaluation of a COVID-19 IgM and IgG rapid test; an efficient tool for assessment of past exposure to SARS-CoV-2 Detection of IgM and IgG antibodies in patients with coronavirus disease 2019 Clinical evaluation of an immunochromatographic IgM/IgG antibody assay and chest computed tomography for the diagnosis of COVID-19 Diagnostic accuracy of an automated chemiluminescent immunoassay for anti-SARS-CoV-2 IgM and IgG antibodies: an Italian experience Diagnostic value and dynamic variance of serum antibody in coronavirus disease 2019 Dynamics of anti-SARS-Cov-2 IgM and IgG antibodies among COVID-19 patients Development and clinical application of a rapid IgM-IgG combined antibody test for SARS-CoV-2 infection diagnosis Evaluation of nucleocapsid and spike protein-based ELISAs for detecting antibodies against SARS-CoV-2 Antibody responses to SARS-CoV-2 in patients with COVID-19 IgA-Ab response to spike glycoprotein of SARS-CoV-2 in patients with COVID-19: a longitudinal study Analytical performances of a chemiluminescence immunoassay for SARS-CoV-2 IgM/ IgG and antibody kinetics Serological immunochromatographic approach in diagnosis with SARS-CoV-2 infected COVID-19 patients Serological assays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Profile of IgG and IgM antibodies against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) Clinical evaluation of a rapid colloidal gold immunochromatography assay for SARS-Cov-2 IgM/IgG Association of viral load with serum biomakers among COVID-19 cases Lower detection rates of SARS-COV2 antibodies in cancer patients vs healthcare workers after symptomatic COVID-19 Reliability and usefulness of a rapid IgM-IgG antibody test for the diagnosis of SARS-CoV-2 infection: a preliminary report Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study Long-term coexistence of SARS-CoV-2 with antibody response in COVID-19 patients Antibody detection and dynamic characteristics in patients with COVID-19 Characteristics of patients with coronavirus disease (COVID-19) confirmed using an IgM-IgG antibody test Significance of Serology Testing to Assist Timely Diagnosis of SARS-CoV-2 infections: implication from a Family Cluster Different longitudinal patterns of nucleic acid and serology testing results based on disease severity of COVID-19 patients Antibodies in infants born to mothers with COVID-19 pneumonia Longitudinal change of SARS-Cov2 antibodies in patients with COVID-19 Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease 2019 Detection of serum IgM and IgG for COVID-19 diagnosis Unintended consequence: influenza plunges with public health response to COVID-19 in Singapore Tracking COVID-19 Performance of 2 commercial serologic tests for diagnosing Zika virus infection Diagnostic accuracy of serological tests for COVID-19: systematic review and meta-analysis Antibody tests for identification of current and past infection with SARS-CoV-2 Reducing false negative PCR test for COVID-19 False-negative results of initial RT-PCR assays for COVID-19: a systematic review Estimating the false-negative test probability of SARS-CoV-2 by RT-PCR Additional supporting information may be found online in the Supporting Information section.