key: cord-344581-h7ikjgic
authors: Ong, David S.Y.; de Man, Stijn J.; Lindeboom, Fokke A.; Koeleman, Johannes G.M.
title: Comparison of diagnostic accuracies of rapid serological tests and ELISA to molecular diagnostics in patients with suspected COVID-19 presenting to the hospital
date: 2020-06-02
journal: Clin Microbiol Infect
DOI: 10.1016/j.cmi.2020.05.028
sha: 
doc_id: 344581
cord_uid: h7ikjgic

OBJECTIVES: To assess the diagnostic performance of rapid lateral flow immunochromatographic assays (LFAs) compared to an enzyme-linked immunosorbent assay (ELISA) and nucleic acid amplification tests (NATs) in suspected coronavirus disease 2019 (COVID-19) patients. METHODS: Patients presenting to a Dutch teaching hospital were eligible between March 17 and April 10, 2020, when they had respiratory symptoms that were suspected for COVID-19. The performances of six different LFAs were evaluated in plasma samples obtained on corresponding respiratory sample dates of NATs testing. Subsequently, the best performing LFA was evaluated in 228 patients and in 50 sera of a historical patient control group. RESULTS: In the pilot analysis sensitivity characteristics of LFA were heterogenous ranging from 2/20 (10%; 95% confidence interval (CI) 0-23) to 11/20 (55%; 95% CI 33-77). In the total cohort, Orient Gene Biotech COVID-19 IgG/IgM Rapid Test LFA had a sensitivity of 43/99 (43%; 95% CI 34-53) and specificity of 126/129 (98%; 95% CI 95-100). Sensitivity increased to 31/52 (60%; 95% CI 46-73) in patients with at least seven days of symptoms, and to 21/33 (64%; 95% CI 47-80) in patients with C-reactive protein (CRP) >100 mg/L. Sensitivity and specificity of Wantai SARS-CoV-2 Ab ELISA was 59/95 (62%; 95% CI 52-72) and 125/128 (98%; 95% CI 95-100) in all patients, respectively, but sensitivity increased to 38/48 (79%; 95% CI 68-91) in patients with at least seven days of symptoms. CONCLUSIONS: There is large variability in diagnostic test performance between rapid LFAs, but overall limited sensitivity and high specificity in acutely admitted patients. Sensitivity improved in patients with longer existing symptoms or high CRP. LFAs should only be considered as additional triage tools when these may lead to the improvement of hospital logistics.

In December 2019 the outbreak of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) started in Wuhan in China [1] , but the coronavirus disease 2019 rapidly spread to other countries as well [2] . The first infected patient in the Netherlands was detected on the 27 th of February 2020 [3] . Accurate diagnostics are fundamental in the fight against this increasing pandemic. Moreover, hospitals would benefit from rapid detection of this virus infection in patients who acutely present to hospitals with respiratory symptoms suspected for COVID-19. Time delay in the establishment of diagnosis increases logistic challenges and causes stagnation of patient flow in emergency departments as these patients are unable to be transferred to appropriate hospital wards or intensive care units (ICUs) when results of the diagnostic tests are still pending [4] .

Nucleic acid amplification tests (NATs) are the gold standard because of the high specificity, although sensitivity may depend on the timing of disease presentation, sampling location and severity of illness [5] . Nevertheless, it usually takes about 4 to 24 hours before laboratory-based results become available depending on specific NAT platforms and laboratory organisation. Therefore, numerous lateral flow immunochromatographic assays (LFAs) have been introduced into the market, and some countries have stocked up on such rapid tests. These LFAs detect the presence of IgM and IgG against SARS-CoV-2.

This study aimed to assess the diagnostic performance of LFAs, and compare these to an enzyme-linked immunosorbent assay (ELISA) and NATs in suspected COVID-19 patients.

Patients presenting to a teaching hospital in the Netherlands were eligible between March 17, 2020 and April 10, 2020 when they had respiratory symptoms that were suspected for respiratory tract infection. Patients were sampled from the oral cavity and subsequently from the nasal cavity using the same nasopharyngeal swab, which was tested by NATs. In some cases, sputum samples were tested, because of persisting clinical suspicion on COVID-19 despite a negative NAT on nasopharyngeal swabs. NATs were performed according to the national reference method that was established after international collaboration [6] , or by the CE-IVD kit GeneFinderTM COVID-19 Plus RealAmp Kit using the Sample to Result Platform ELITe InGenius®. The Institutional Review Board waived the need for informed consent as tests were performed on samples which had been acquired for routine clinical care (IRB protocol number 2020-034), and according to hospital procedure all patients were informed about the possibility of an opt-out if they had objections against the use of left-over material for research to improve or validate diagnostic testing procedures. The study was conducted in accordance with Helsinki Declaration as revised in 2013.

First, in a pilot phase 20 NAT-positive and 5 NAT-negative patients were retrospectively selected for which six LFAs were performed on heparin plasma samples obtained upon hospital presentation ( Figure S1 ), which corresponded to the dates of molecular testing. LFAs were included from Boson Biotech, Cellex, Dynamiker Biotechnology, Orient Gene Biotech, Prometheus Bio, and Wantai Rapid Test. Any visible band for either IgG, IgM or unspecified Ig was indicative for a positive result. Second, based on the sensitivity and specificity results in the pilot study, the best performing LFA was further evaluated in an extended cohort of randomly selected patients. Third, this LFA was prospectively tested in consecutive patients between April 6 and April 10. Fourth, specificity was additionally tested in a historical control group of randomly selected sera of 50 adult patients in September 2019 as SARS-CoV-2 was not circulating at that time. Finally, samples were also analysed by the Wantai SARS-CoV-2 Ab ELISA kit, which detects total antibodies, and interpreted according to manufacturer's instructions. Both clinical information and reference standard results were unavailable to the performers of LFAs and the ELISA.

All analyses were performed using SAS 9.2 (Cary, North Carolina). We compared groups using non-parametric tests for continuous variables and Chi-square test or Fisher's Exact Test for categorical variables as appropriate. P-values <0.05 were considered to be statistically significant.

In the pilot study sensitivity characteristics of LFA were very heterogenous ranging from 2/20 (10%; 95% confidence interval (CI) 0-23%) to 11/20 (55%; 95%CI 33-77%)) ( Table 1) . We decided to continue with the Orient Gene Biotech COVID-19 IgG/IgM Rapid Test (OGBRT) as it had the highest sensitivity.

A total of 111 patients (including the 25 from the pilot study) were retrospectively selected between March 16 and March 29. Subsequently, 117 consecutive patients were prospectively included between April 6 and April 10. In total, 228 patients were included with a median age of 61 years (interquartile range (IQR) 46-74), 117 (52%) were male, 21 (9%) were admitted to the ICU within 24 hours and median C-reactive protein (CRP) upon hospital presentation was 31 (IQR 7-95) mg/L (Table S1 ). Median time from symptom onset to sample collection was 7 (IQR 4-14) days.

OGBRT had an overall sensitivity of 43/99 (43%; 95%CI 34-53%) and specificity of 126/129 (98%; 95%CI 95-100%) ( Table 2) . Sensitivity increased to 31/52 (60%; 95%CI 46-73) in patients with at least seven days of symptoms, and to 21/33 (64%; 95%CI 47-80) in patients with CRP >100 mg/L upon presentation. However, there was no significant difference between patients requiring ICU Figure S2 ).

In the randomly selected historical control sera, the LFA and the ELISA specificity was 49/50 (98%; 95%CI 94-100) and 50/50 (100%; 95%CI 100-100), respectively; LFA showed a very weak IgG line in one sample.

This study shows that the sensitivity of LFA was low in patients suspected for COVID-19 presenting to the hospital, but it improved in patients with at least seven days of symptoms and in those with CRP levels >100 mg/L upon presentation. Specificity of LFAs and the ELISA was very high, and fulfilling a frequently used criterium of at least 98%. The ELISA had a higher sensitivity compared to LFAs.

Several countries, including Spain and the United Kingdom, have purchased one or more of these LFAs. However, our study findings underline that cautiousness is required when considering implementation of such tests. Interestingly, Cellex Rapid Test, which is currently the only rapid diagnostic test that is FDA approved, performed less than OGBRT in our pilot study. Another rapid test was reported to have a sensitivity below 20% in acute patients referred to emergency department [7] . Other studies showed higher sensitivities of LFAs up to 90% in unspecified patient groups with more time between disease onset and testing or missing information regarding timing of sampling [8, 9] . Test performance characteristics as provided by manufacturers were higher than those observed in our study, which is related to different selection of positive and negative controls.

In our study we primarily included consecutive patients presenting to the hospital, which represents clinical practice and clinical sensitivity (i.e. diagnosing COVID-19 upon hospital presentation) rather than analytical sensitivity (i.e. detecting the presence of antibodies at that moment). The observed higher sensitivity in patients with at least seven days of symptoms is in line with findings from other studies [10, 11] .

There are some study limitations to consider. This study included a wide comparison of six different LFAs, an ELISA and NATs, but more tests are available on the market. Nevertheless, both

LFAs and the ELISA were limited in sensitivity, suggesting that antibody production is not always detectable or at least not yet detectable during the early phase of infection. Second, NAT as reference standard remains suboptimal, and it remains possible that in some cases actual infections were missed. In some patients NATs were only positive in sputum and negative in nasopharynx, whereas the majority of patients were only tested by nasopharyngeal swabs. Third, the subgroup of patients admitted to the ICU was limited, precluding definite conclusions in this group.

In conclusion, the high specificity of LFAs may contribute to rapidly confirm COVID-19, accelerate decision-making in emergency rooms and routing to appropriate hospital wards. Yet, negative LFA results are unreliable to exclude COVID-19 due to the limited sensitivity of these tests. Therefore, these LFA tests cannot replace molecular diagnostics in acute care settings, but should only be used as an additional triage tool when improvement of hospital logistics is expected and their limitations are carefully considered.

The authors declare no conflicts of interest. 

Clinical Characteristics of Coronavirus Disease 2019 in China

Rapidly increasing cumulative incidence of coronavirus disease (COVID-19) in the European Union/European Economic Area and the United Kingdom

Current information about the novel coronavirus (COVID-19) Bilthoven: RIVM

Better tests, better care: improved diagnostics for infectious diseases

SARS-CoV-2 Viral Load in Upper Respiratory Specimens of Infected Patients

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

Performance of VivaDiag COVID-19 IgM/IgG Rapid Test is inadequate for diagnosis of COVID-19 in acute patients referring to emergency room department

Evaluation of a COVID-19 IgM and IgG rapid test; an efficient tool for assessment of past exposure to SARS-CoV-2

Development and clinical application of a rapid IgM-IgG combined antibody test for SARS-CoV-2 infection diagnosis

Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease

Antibody Detection and Dynamic Characteristics in Patients with COVID-19

Boson Biotech Rapid 2019-nCoV IgG/IgM Combo Test Card 10 / 20 (50%

Cellex qSARS-CoV-2 IgG/IgM Cassette

Dynamiker Biotechnology 2019-nCOV IgG/IgM

Orient Gene Biotech COVID-19 IgG/IgM Rapid Test Cassette 11 / 20 (55%

Prometheus Bio 2019-nCOV IgG/IgM

In the subgroup of patients with time from symptom onset to sample collection >= 14 days, sensitivity and specificity of the LFA were 9/14 (64%; 95%CI 39-89) and 30/32 (94%; 95%CI 85-100), respectively, whereas sensitivity and specificity of the ELISA

Of note, sensitivity was 17/24 (71%) in patients with >=7 days from symptom onset to sample collection and CRP >=100 mg/L

We would like to thank our laboratory technicians and team managers for their assistance in performing the serological tests.Contribution: DSYO and JGHK contributed to the conception and design of the study. DSYO, SJM and FAL acquired the data. DSYO and SJM analysed the data. All authors contributed to the interpretation of the data. DSYO drafted the first manuscript and all other authors revised it critically for important intellectual content. All authors approved this manuscript version to be submitted.