key: cord-0935614-16l1189y
authors: Clark, Tristan W.; Brendish, Nathan J; Poole, Stephen; Naidu, Vasanth V.; Mansbridge, Christopher; Norton, Nicholas; Wheeler, Helen; Presland, Laura; Ewings, Sean
title: Diagnostic accuracy of the FebriDx host response point-of-care test in patients hospitalised with suspected COVID-19
date: 2020-06-21
journal: J Infect
DOI: 10.1016/j.jinf.2020.06.051
sha: 4a51b31dcbfffccdc204420e8050dc158982e13d
doc_id: 935614
cord_uid: 16l1189y

INTRODUCTION: Management of the COVID-19 pandemic is hampered by long delays associated with centralised laboratory PCR testing. In hospitals this leads to poor patient flow and nosocomial transmission and so rapid, accurate diagnostic tests are urgently required. The FebriDx is a point-of-care test that detects an antiviral host response protein in finger prick blood within 10 minutes, but its accuracy for the identification of COVID-19 is unknown. METHODS: We performed a real-world diagnostic accuracy study of FebriDx in hospitalised patients during the first wave of the pandemic. Measures of diagnostic accuracy were calculated based on FebriDx results compared to the reference standard of SARS-CoV-2 PCR on combined nose and throat swabs. A multivariable predictive model including FebriDx, age, sex, and clinical characteristics was developed and underwent internal validation. RESULTS: FebriDx was performed on 251 patients and gave a valid result in 248. 118 of 248 (48%) were PCR positive for COVID-19. FebriDx results were available after 10 minutes compared with 1.7 (1.6 to 2.1) hours with point-of-care PCR testing and 23.4 (17.2 to 31.1) hours with laboratory PCR testing. Sensitivity of FebriDx for the identification of COVID-19 was 93% (110/118; 95% CI 87 to 97%) and specificity was 86% (112/130; 95%CI 79 to 92%). Positive and negative likelihood ratios were 6.73 (95%CI 4.37 to 10.37) and 0.08 (95%CI 0.04 to 0.15) respectively. In the multivariate model age, sex and other clinical features did not contribute significantly to the effect of the FebriDx result in distinguishing patients with and without COVID-19. CONCLUSIONS: During the first wave of the pandemic, FebriDx had high accuracy for the identification of COVID-19 in hospitalised adults and could be deployed as a front door triage tool. TRIAL REGISTRATION: ISRCTN14966673

The management of COVID-19 in secondary care is severely hampered by the long turnaround times of centralised laboratory PCR testing, which can take up to several days to generate results. In acute hospitals this leads to poor patient flow though acute areas, as suspected patients are held in assessment cohort areas until their results are available. In addition, lack of single occupancy rooms means that COVID-19 negative patients cohorted in these assessment areas may acquire infection from positive patients before test results are available. Rapid, accurate diagnostic tests are therefore urgently required. Molecular point-of-care testing (POCT) for COVID-19 may mitigate this situation but regulatory requirements for new POCT assays and difficulties in rapidly upscaling manufacture mean that the availability of these tests is currently severely limited. Alternative diagnostic solutions are therefore urgently required.

FebriDx (Lumos diagnostics, Sarasota, Florida, US) is a CE-marked POCT that detects two host response proteins, Myxovirus resistance protein A (MxA) and C reactive protein (CRP), in finger prick blood samples, and is designed to distinguish viral from bacterial respiratory infection. [1] [2] [3] [4] [5] MxA is a marker of interferon-induced antiviral host response and in our previous work, the detection of MxA by FebriDx had high sensitivity for the detection of Influenza in hospitalized adults, during influenza season. 6 MxA levels are also likely to be significantly elevated in patients with COVID-19 but the diagnostic accuracy of FebriDx in this situation is currently unknown. FebriDx is a low cost, instrument-free, disposable POCT system and if sensitive for the detection of COVID-19 could be rapidly deployed across the NHS and other healthcare systems to help mitigate the long delays in diagnostic testing and provide a rapid triage tool. The aim of this study was therefore to evaluate the real-world diagnostic accuracy of MxA detection by FebriDx for the identification of COVID-19 in hospitalised adults during the first wave of the pandemic, and to develop and internally validate a multivariable model for diagnosis of COVID-19 that includes FebriDx result and other easily obtained clinical measures.

This study of FebriDx was nested within the CoV-19POC study, a trial assessing the clinical impact of molecular POCT for COVID-19. Adults presenting with suspected COVID-19 were enrolled; full details of the inclusion and exclusion criteria can be found in the protocol, linked below. The study was approved by the South Central -Hampshire A Research Ethics Committee: REC reference 20/SC/0138, on the 16th March 2020. The protocol is available at: https://eprints.soton.ac.uk/439309/1/CoV_19POC_Protocol_v1.1_eprints.pdf

Combined nose and throat swabs were obtained from patients by research staff and tested immediately on the QIAstat-Dx platform using the Respiratory SARS-CoV-2 Panel, 7 at the point of care. A full list of the pathogens detected by the panel is shown in Table E1 in the online data supplement. 8, 9 In addition, laboratory PCR testing for SARS-CoV-2 on combined nose and throat swabs was performed on all patients, in the on-site Public Health England 6 (PHE) microbiology laboratory, using the PHE RdRp and E gene reference assays. 10, 11 Demographic and clinical data was collected at enrolment and outcome data collected retrospectively from case note and electronic systems, using an electronic case report form.

For this sub-study the first 266 patients enrolled were consecutively approached for testing using the FebriDx host response POCT on finger prick blood samples, taken at the same time as the nose as throat swabs for PCR. Detailed instructions for use of FebriDX are provided via this link: https://www.febridx.com/how-to-use#testing but in brief; after puncture of the skin with the integral lancet, 5 microliters of blood are drawn into the blood collection tube via capillary action by placing it against a blood drop. The blood is then transferred to the lateral flow section of the device and reagents released by pressing a button. The test is read after 10 minutes.

The FebriDx system generates results in the form of the presence or absence of three lines, assessed by visual inspection; the CRP line (top, grey), the MxA line (middle, red) and the control line (bottom, blue). Figure 2 shows examples of FebriDx positive and negative tests.

The colour change on the CRP and MxA lines when detected is dependent on the amount of the target protein in the sample and so there is potential variability in interpretation with weaker lines (quoted threshold for detection of CRP line 20mg/L and MxA line of 40ng/ml).

For this reason each FebriDx result was read independently by two investigators and if there was disagreement on the results displayed this was further adjudicated by a third investigator. The results of the FebriDx were not shared with the clinical teams and the readers of the FebriDx test lines were blinded to the PCR results (FebriDx results were read and recorded before PCR results were available).

For the reference standard of PCR, the QIAstat-Dx PCR system gives a readout of positive or negative for the detection of targets including SARS-CoV-2. 7 The PHE laboratory RdRp and E gene assay PCR results are also provided as a binary result, RNA detected or RNA not detected, and the detection of RNA by either assay is considered a positive result. For this study PCR detection by either QIAstat-Dx or laboratory PCR or both was considered as positive for COVID-19.

The sample size for this sub-study was driven by consideration for estimating sensitivity of FebriDx MxA for the detection of COVID-19 (as defined by the reference standard of detection of SARS-CoV-2 by PCR on respiratory samples) and of the methods proposed by Riley et al 12 for multivariable predictive models. In order to estimate a sensitivity of 85% to within +/-8%, based on the score method for a 95% confidence interval, 13 88 positive cases are required. Assuming a prevalence of Covid-19 of around 40% in those tested, 220 people are required, and to achieve an 80% chance of obtaining enough cases, this number was increased to 236. This was further increased to 266 to allow for patients who decline or are unable to undergo finger prick testing (estimated from the rates in our previous work). Full justification for the samples size in the multivariable model is provided in the supplementary appendix.

Baseline characteristics are summarised for all those recruited to the study, where data are available, and presented for the whole sample and by COVID-19 status. Diagnostic accuracy was assessed in two ways. Simple measures of sensitivity, specificity, predictive values and likelihood ratios are given for FebriDx MxA detection for the identification of COVID-19 (and as FebriDx MxA is a broad marker of antiviral host response and not limited to COVID-19 these measures are also calculated for the detection of all viruses). To further assess the diagnostic value of the test in practice, a multivariable logistic regression model was developed, 14, 15 full details are provided supplementary appendix.

As the antiviral host response to SARS-CoV-2 would be expected to wane over time, but detection of viral RNA by PCR may persist for several weeks, a secondary analysis was prespecified to assess diagnostic accuracy in those who have had duration of less than seven days. Basic measures of diagnostic accuracy (e.g., sensitivity, specificity) are presented separately for those with duration less than or more than seven days. A multivariable model, pre-specified to be the same as that for the full model, was also run in those with duration less than seven days.

This study was prospectively registered with the ISRCTN14966673 on the 18 th March 2020.

Between the 20th March and 12th April 2020 266 patients consecutive patients recruited to the CoV-19POC study were approached and FebriDx was performed in 251. 12 of 251 (5%) tests failed to give a valid result on the first test and a subsequent valid result was obtained on re-testing in 9 of 12, so that overall 248 patients had a valid FebriDx result. Flow of study participants is shown in Figure 1 were less likely to be ethnically White British, had been unwell for longer (median duration of illness prior to presentation of 8 days), had a higher temperature and respiratory rate and lower oxygen saturations, compared to COVID-19 negative patients. In addition, COVID-19 positive patients had higher CRP levels, lower white cell counts and were more likely to have radiological evidence of pneumonia, shown in Table 1 .

Compared to the reference standard of detection of SARS-CoV-2 RNA by PCR on respiratory samples, the sensitivity of FebriDx MxA detection for identification of COVID-19 was 93% (110/118; 95%CI 87 to 97%). Specificity in this cohort was 86% (112/130; 95%CI 79 to 92%).

Positive predictive value (PPV) was 86% (110/128; 95% CI 79 to 91%) and negative predictive value (NPV) was 93% (112/120; 95% CI 87 to 97%). Overall accuracy was 90% (222/248; 95%CI 86 to 93%), shown in Table 2 . As detection of MxA is not specific for COVID-19 the diagnostic accuracy of FebriDx MxA detection for the identification of all respiratory viruses is shown in Table 3 . Of the 11 patients with positive FebriDx MxA but who were PCR negative for SARS-CoV-2 and other viruses, 7 of 11 (63%) had classic radiological features of COVID-19.

Using the calculated performance of FebriDx MxA for the detection of COVID-19 to estimate PPV and NPV at different disease prevalence, a prevalence of COVID-19 of 20% would give a PPV of 63% (95%CI 52 to 72%) and NPV of 99% (95%CI 96 to 99%). At a prevalence of 10% the PPV would be 43% (95%CI 33 to 53%) and the NPV would be 99% (95%CI 98 to 100%).

Measures of diagnostic accuracy according to duration of symptoms are shown in Table 4 .

Although direct comparison of accuracy was not possible due to samples size, the sensitivity, specificity and positive and negative predictive values of FebriDx appeared comparable for patients presenting before and after 7 days of symptom duration.

A total of 201 participants contributed data to the multivariable model, with symptoms being the main contributor to missing data. With the possible exception of cough (12% missing data in those without COVID-19 versus 6% with), missingness of data appeared unrelated to outcome. All continuous variables (age, respiratory rate and temperature) were mean-centred and assessed for a non-linear relationship with outcome using restricted cubic splines, following visual assessment and using Bayesian Information Criterion values (where smaller values indicate a more parsimonious model), all were entered in the model on the assumption that a linear relationship with the log odds of having COVID-19 was appropriate. All variables were entered into the multivariable model with no variable selection. Model results are given in Table 5 , alongside adjusted estimates.

The overall results suggested that the addition of the patient and clinical characteristics do not add significantly to the diagnostic accuracy compared to using FebriDx MxA result alone. 

The multivariable model was rerun with anosmia; a total of n=164 were included in this analysis. Conclusions for the other covariates did not change, and there was little change in discrimination results (area under the curve, sensitivity and specificity at the Youden index, Brier score) nor other measures of fit such as Nagelkerke's R 2 ; anosmia itself had an estimated odds ratio of 4.0 (95% CI: 0.7 to 22; p=0.11) suggesting a potentially useful diagnostic indicator, though this conclusion is hampered by the uncertainty in the estimate.

Lastly, the multivariable model (excluding anosmia) was also run in the subgroup of participants who presented within 7 days of symptom onset. Although the odds ratio for FebriDx increased (190), the uncertainty of this effect was large (95% CI 26 to 1404). There was little change in model performance with regard discrimination and model fit.

As detailed above, some FebriDx tests failed to give a valid result on the first attempt. Tests could fail by blood clotting in the sample collection tube so that no blood would pass onto the lateral flow strip.

There were no adverse events seen with FebriDx testing. A single patient suffered brief epistaxis following nasal swabbing for PCR testing.

In this large, real-world study we have shown that that the detection of the antiviral host response protein MxA using the FebriDx point-of-care test had high sensitivity for the identification of COVID-19 during the first wave of the pandemic. Although there were differences in baseline characteristics between COVID-19 positive and negative patients, the multivariate predictive model results showed that clinical features did not contribute significantly to the effect of the FebriDx MxA result in distinguishing these groups. MxA detection is a marker of antiviral host response and is not specific for SARS-CoV-2 and so the high specificity of FebriDx MxA for COVID-19 seen in this study reflects the low levels of other respiratory viruses circulating during the study which will change with the seasonal circulation of other viruses such as influenza. It is now recognised that the diagnostic accuracy of PCR on upper respiratory tract samples is suboptimal and that patients presenting with COVID-19 often initially test falsely negative when compared to serial PCR testing or CT scanning. [16] [17] [18] This means that our reference standard was likely to be suboptimal and it is notable that several of the patients in our study with positive MxA results and negative PCR for viruses (ie those considered to be false positive MxA) had classical radiological changes of COVID-19, strongly suggesting that they had the disease despite negative PCR results. The calculated specificity of FebriDx MxA for the detection of COVID-19 in the first wave was therefore likely to have been an underestimate.

The current reliance on centralised laboratory PCR for the diagnosis of COVID-19 with its prolonged turnaround time, leads to long delays in identification of positive and negative patients, with subsequent nosocomial transmissions, poor patient flow and reduced operational capacity in hospitals. As FebriDx is a low cost (£12 per test), rapid and highly scalable test these findings suggest that FebriDx could be rapidly deployed in hospitals and urgent care centres to be used as a front door triage tool. Because of the high sensitivity and negative predictive value, MxA negative patients being admitted to hospitals could be rapidly directed to non COVID-19 areas whilst MxA positive patients could be moved to assessment areas whilst awaiting confirmation or exclusion of COVID-19 positive status by PCR or CT scanning.

The manufacturers of FebriDx state that it is intended for use in patients aged over two years of age who have had respiratory symptoms for less than or equal to 7 days and/or fever for 48 hours duration or less. In this study we have demonstrated that MxA detection by FebriDx remains accurate for the detection of COVID-19 beyond this period, presumably due to a very strong and persistent antiviral host response to SARS-CoV-2.

There are several limitations to the generalisability of our study. As the study was performed in almost exclusively immunocompetent adults the results cannot be applied to immune-compromised patients or to children and separate evaluations should be urgently undertaken in these important patient groups. In addition the findings of our study cannot be extrapolated to community dwelling patients including those who are infected but asymptomatic or pauci-symptomatic, as it uncertain whether their antiviral host response would be comparable to hospitalised patients. The diagnostic accuracy of FebriDx reported in this study is in the context of a very high disease prevalence and as we have shown the performance characteristics will change as the disease prevalence drops, with implications for how the test could be used. In particular the positive predictive value will decrease now that the first wave is subsiding, although the negative predictive value will increase further meaning that a negative FebriDx will remain a useful rule-out test. As MxA detection is a marker of antiviral host response and is not specific for COVID-19, the calculated test specificity will be lower when other viruses such as influenza are circulating. As early identification of influenza alongside COVID-19 will remains vitally important in hospitals, a rapid triage tool that detects both influenza and COVID-19 is likely to be of utility in the coming winter months. Finally, as this was a single centre study the diagnostic accuracy of FebriDx MxA for COVID-19 should be external validated in other centres.

In summary, FebriDx MxA had high accuracy for the identification of COVID-19 during the first wave of the pandemic and could be rapidly deployed in secondary care settings as a real-time triage tool to address the current problems of delayed diagnosis with laboratory PCR.

TWC reviewed the medical literature, conceived of and designed the study, oversaw the conduct of the study, participated in the analysis and interpretation or results, drafted and wrote the manuscript. NJB assisted with the design of the study, screened and recruited patients and collected data. SP, CM, NN, VN, HW and LP screened and recruited patients and collected data. SE analysed the data and created and internally validated the multivariate model. All authors reviewed and contributed to the manuscript during its development. (21) 78/129 (60) -All data are given as n/n (%) and median (inter-quartile range). COVID-19 positive and negative status is defined by PCR positivity or negativity for SARS-CoV-2. *P value relate to comparison between COVID-19 positive and negative patients. NEWS=National Early Warning Score 2, CRP=C reactive protein, WCC=White cell count. †For the most recent influenza season. ‡Duration of symptoms prior to presentation, evaluated in 215 patients. 

MxA-based recognition of viral illness in febrile children by a whole blood assay

Diagnosis of viral infections using myxovirus resistance A (MxA)

Diagnostic accuracy of FebriDx: A rapid test to detect immune responses to viral and bacterial upper respiratory infections

A prospective, multi-centre US clinical trial to determine accuracy of FebriDx point-of-care testing for acute upper respiratory infections with and without a confirmed fever

A Rapid Diagnostic Test for Differentiating Bacterial and Viral Aetiologies in Acute Respiratory Infections

Evaluation of the Febridx Host Response Point-of-Care Test to Differentiate Viral From Bacterial Aetiology in Adults Hospitalized with Acute Respiratory Illness During Influenza Season

Multicenter evaluation of the QIAstat Respiratory Panel-A new rapid highly multiplexed PCR based assay for diagnosis of acute respiratory tract infections

Multicenter Evaluation of the QIAstat-Dx Respiratory Panel for the Detection of Viruses and Bacteria in Nasopharyngeal Swab Specimens

Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR

Minimum sample size for developing a multivariable prediction model: PART II -binary and time-to-event outcomes

Two-sided confidence intervals for the single proportion: comparison of seven methods by

Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal

Regression modelling strategies for improved prognostic prediction

Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases

Diagnostic performance between CT and initial real-time RT-PCR for clinically suspected 2019 coronavirus disease (COVID-19) patients outside Wuhan, China

Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure

We would like to acknowledge and gives thanks to all the patient who kindly participated in this study and to all the clinical staff at University Hospital Southampton who cared for them.We would also like to give thanks to Richard Poole for assistance in data management. This report is independent research supported by the National Institute for Health Research