key: cord-1053574-oi7x89ca
authors: Levine-Tiefenbrun, M.; Yelin, I.; Uriel, H.; Kuint, J.; Schreiber, L.; Herzel, E.; Katz, R.; Ben-Tov, A.; Patalon, T.; Chodick, G.; Kishony, R.
title: Association of COVID-19 RT-qPCR test false-negative rate with patient age, sex and time since diagnosis
date: 2020-11-03
journal: nan
DOI: 10.1101/2020.10.30.20222935
sha: 31705ca1fa929ea6c5746545100e08d8cc87aef2
doc_id: 1053574
cord_uid: oi7x89ca

Background Routine testing for SARS-CoV-2 in the community is essential for guiding key epidemiological decisions from the quarantine of individual patients to enrolling regional and national preventive measures. Yet, the primary testing tool, the RT-qPCR based testing, is notoriously known for its low sensitivity, i.e. high risk of missed detection of carriers. Quantifying the false-negative rate (FNR) of the RT-qPCR test at the community settings and its dependence on patient demographic and disease progression is therefore key in designing and refining strategies for disease spread prevention. Methods Analyzing 843,917 test results of 521,696 patients, we identified false-negative (FN) and true-positive (TP) results as negative and positive results preceded by a COVID-19 diagnosis and followed by a later positive test. Regression analyses were used to determine associations of false-negative results with time of sampling after diagnosis, patient demographics and viral loads based on RT-qPCR Ct values of the next positive tests. Findings The overall FNR was 22.8%, which is consistent with previous studies. Yet, this rate was much lower at the first 5 days following diagnosis (10.7%) and only increased in later dates. Furthermore, the FNR was strongly associated with demographics, with odds ratio of 1.74 (95% CI: 1.58-1.9) for women over men and 2.54 (95% CI: 2.39-2.69) for a 20 versus a 50 year old patient. Finally, FNR was associated with viral loads (p-value 0.002), with a difference of 1.1 (95% CI: 0.60-1.57) between the average Ct of the N gene in a positive test following a false-negative compared to a positive test following a true-positive. Interpretation Our results show that in the first few days following diagnosis, when results are critical for quarantine decisions, RT-qPCR testing is more reliable than previously reported. Yet the reliability of the test result is reduced in later days as well as for women and younger patients, where the viral loads are typically lower. Funding This research was supported by the ISRAEL SCIENCE FOUNDATION (grant No. 3633/19) within the KillCorona-Curbing Coronavirus Research Program.

The ongoing COVID-19 pandemic has already infected more than 45 million people worldwide ( https://coronavirus.jhu.edu/map.html , October 30th, 2020 1 ). A major tool in combating the pandemic is testing for viral carriage, which is used for both diagnostic and epidemiologic purposes. The most commonly used viral detection tests are based on the reverse transcription quantitative polymerase chain reaction of viral genes (RT-qPCR). This nucleic acid test is of high specificity, i.e. very low false-positive rate [2] [3] [4] [5] . In contrast, a high false-negative rate was reported for these tests [6] [7] [8] [9] [10] . These high false-negative rates impede local and global efforts to slow down disease spread, as patients incorrectly diagnosed as non-carriers may subsequently infect additional people 11 . Systematically quantifying the rate of false-negative results and its dependencies on disease progression and patient demographics is critical for disease spread modeling, public health policy making and person-level quarantine decisions.

Various approaches have been taken to estimate the false-negative rate of COVID-19 RT-qPCR tests. Measuring the rate of false-negative results in a population of patients with highly specific pathologies ( e.g. chest CT), has initially alerted physicians and epidemiologists of the high false-negative rate, estimated at approximately 30% [4] [5] [6] [7] 9 . A meta-analysis of multiple such studies found that the reported rates were highly variable with a mean false-negative rate of 11% 12 . However, and as previously noted 12, 13 , these meta-analysis studies were necessarily based on a combination of variable studies of non-uniform origins and methodologies, typically involving small groups of patients. A more recent systematic approach was based on 'longitudinal testing' in which the accuracy of each test is determined based on later tests of the same patient: a negative test which is directly followed by a positive one is deemed false negative. Application of this approach in a hospital setting resulted in an estimation of a false-negative rate of 17.8% 13 . Systematic large scale studies at the community, critical for epidemiological disease control, have been lacking.

Beyond the average false negative rate, it is also important to understand whether and how the false-negative rate is associated with patient-specific and sample-specific attributes. Meta-analysis studies showed a strong association of false-negative results with time since exposure 14 or time since onset of symptoms 15 . At the patient specific level, as viral load is associated with time since onset of symptoms, sex and age [16] [17] [18] [19] [20] [21] [22] , it has been proposed that false-negative rates might also depend on demographics, but current studies lacked statistical power for quantifying such dependencies 23 .

Here, we apply a longitudinal testing based approach to a large dataset of patient-level test series with linked demographics and electronic health records, to qunatify the false-negative rate of COVID-19 test results at the community and its associations with age, sex and time since diagnosis. Finally, we test whether the risk of false-negative results is associated with viral load at the single-patient level.

Anonymized clinical records of SARS-CoV-2 RT-qPCR test results (test reports) were retrieved by Maccabi Healthcare Services (MHS) for the period between February 8th and September 24th 2020. Records of COVID-19 or COVID-19-related diagnoses by physicians (diagnosis reports) and referrals based on suspected exposure to the disease (epidemiological-based referrals) were retrieved for these patients. When available, fluorescence measurements data of the PCR test were retrieved for each test ( RT-qPCR measurements ). Randomly generated identifiers were used to link between test results and diagnosis codes.

Test results. MHS aggregates all test results for all its members, whether or not the test itself was performed by MHS laboratory. Test results data included, for each test: random patient number, sample number, sample execution date and test result. Test results were either "positive" (7.4%), "negative" (92%) or "borderline-positive" (0.6%, which we considered as positive in our analysis). Patients for whom two tests with different results were recorded on the same day were excluded from the analysis (274 patients, 0.05%).

Diagnosis reports. Diagnoses are routinely recorded in MHS database. For all patients with at least one positive test result, we retrieved any symptom-based COVID-19 diagnoses recorded prior to their first test. Diagnosis report data included: random patient number, diagnosis date, and diagnosis code (ICD9 and internal MHS codes).

Epidemiologically-based referrals . Since April 3rd 2020, an epidemiologically-based referral was filled by physicians referring a patient to a SARS-CoV-2 RT-qPCR test. Each referral report included: random patient number, referral number, referral date and referral cause.

RT-qPCR measurements. Fluorescence measurements data were retrieved from all 7 Bio-rad CFX96 RT-qPCR machines of MHS central laboratories. For each test, the following data were included: sample number, PCR machine number, test well number, test date, fluorescence measurements for 45 amplification cycles in 4 channels: FAM, Cal Red 610 Quasar 670 and HEX, corresponding to the measurements of E gene, RdRp gene, N gene and the internal control, respectively.

For each patient, the earliest date of COVID-19 symptoms and/or epidemiologically-based referral was considered as "date of diagnosis". When both symptom-based diagnosis and epidemiological-based referrals were available, they were usually recorded on the same day. For simplicity, we excluded a small number of patients for whom both a diagnosis and a referral were available, but were more than a day apart (5.2% of diagnosed patients).

For any patient with at least one positive test, a 'positive period' was defined as the period between their date of diagnosis and their last positive test. Negative test reports during this period were regarded as false-negative (FN), while positive test reports during this period were regarded as true-positive (TP). False-negative rate (FNR) was calculated as .

Logistic regression of a false-negative versus true-positive result was performed using Python's statsmodels library. The probability of a false-negative result was fitted to the test result (true-positive: 1, false-negative: 0) for all tests within the positive period.

Linear regression of Ct values for each fluorescence channel was performed using Python's statsmodels library. 

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; since diagnosis, and , , and are the fitted parameters, with the latter two a b R early R late standing for the false negative rates at the early and late phases, respectively.

According to patient age, test results were divided into two groups of similar size (<40 , ≥40 years). FNR was calculated separately for each group. Statistical significance for differences in FNR between groups was tested using a two-sided Fisher's exact test (SciPy in Python).

Fluorescence measurements of each channel for each well were normalized by the mean measurement of the first 5 PCR cycles. For each sample, Ct was defined for each gene as the PCR cycle in which the normalized fluorescence measurement crossed a set threshold (FAM: 1.1, Cal Red 610: 1.2, Quasar 670: 1.2 and HEX: 1.2).

The study protocol was approved by the ethics committee of Maccabi Healthcare Services, Tel-Aviv, Israel.

Among all~2 million MHS patients, we identified 843,917 recorded tests for 521,696 patients ( is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; https://doi.org/10.1101/2020.10.30.20222935 doi: medRxiv preprint

To identify personalized features associated with false-negative results, we performed multivariate logistic regression for the odds of a false-negative result (Methods: 'Logistic regression' and 'Calculating odds ratio from logistic regression' ). Patient age, sex and number of days from day of diagnosis were all associated with a false-negative result (supplementary table 2). Patient age was strongly and negatively associated with a false-negative result, with odds ratio of 2.54 (95% CI: 2.39-2.69) for a 10 versus a 40 year old patient. The number of days from the day of diagnosis was positively correlated with a false-negative result, with OR of 2.16 (95% CI: 1.96-2.38) for samples taken at day 15 compared to samples taken at day 0. Lastly, patient sex was also associated with false-negative results, with female to male odds ratio of 1.74 (95% CI: 1.58-1.9).

Following the observed association between time after diagnosis and false-negative result, we characterized the FNR during disease progression. Calculating FNR per day after diagnosis (Methods: 'calculating FNR' ), we found that FNR followed 3 distinct phases: at the first few days following diagnosis, it was fairly constant and low (10.7%, days 0-5). It then gradually increased over days 6-15, and finally it plateaued at high rates of about 39% (Methods: 'Curve fitting for FNR over time' ; figure 2A).

We then focused on the earlier days after diagnosis (days 0-5), in which FNR was relatively low, and in which a precise diagnosis is most critical for epidemiological patient-level quarantine control. Multivariate logistic regression analysis of test results for these days alone identified the association of false-negative results during these days with sex and age. A 10 years old versus 40 years old patient has an odds ratio of 3.79 (95% CI: 3.42-4.2), and female patients have an OR of 2.01 (95% CI: 1.69-2.39) compared with males (supplementary table 3 ). Dividing the patients into 2 age groups of similar size (<40 and ≥40, table 1), we found that FNR during this initial period was significantly higher for the younger age group (p-value 0.0002, Fisher's exact test, OR=0.71, 95% CI: 0.58-0.87; figure 2B) . Similarly, the FNR during the later period (days 6-24) also significantly decreased with age (p-value 0.02, Fisher's exact test).

Based on previous reports of viral load differences along disease progression, between males and females and among age groups 5, [16] [17] [18] [19] 21, [29] [30] [31] [32] [33] [34] , we hypothesized that differences in FNR across demographic factors and disease progression may stem from changes in viral load, which would be reflected in the measured Ct values. To test this hypothesis, we first tested for associations of Ct values of the three viral genes (N gene, E gene and RdRp) and the internal control gene (IC) with patient age and sex and number of days after diagnosis ( figure 3, supplementary figure 1 ). Indeed, a linear regression model revealed positive correlation of the Ct of viral genes with the number of days after diagnosis, and negative correlation with age and sex (male; Methods: 'Linear regression' , supplementary table 4). An opposite association was found with the IC gene, in agreement with within-tube competition for reagents between the multiplexed reactions 35 (supplementary figure 1C) . The viral load association with demographics and time, therefore, mirrored the associations found for the FNR. Finally, we tested more directly for association of false-negative rate with viral load at the individual patient level. Since Ct values are not available for false-negative results, we used as a proxy the Ct values of the next positive result. Comparing the distribution of Ct values of positive test results following false-negative tests with, as a control, the Ct values of positive test results following true-positive tests, we found that is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; https://doi.org/10.1101/2020.10.30.20222935 doi: medRxiv preprint indeed false-negative results are associated with reduced viral load for all three viral genes ( figure 4 and supplementary figure 2 ; Mann-Whitney U test; p-value of , , .4 1 * 10 −5 .5 1 * 10 −2 for N, E and RdRp genes, respectively). .8 1 * 10 −3

Our analysis of large dataset of electronic health records of COVID-19 patients showed that while on average the FNR is about 23%, consistent with past measurements, this rate varies strongly with age, sex and time after diagnosis. At the first few days following diagnosis, the FNR is only 10% on average and even lower for men and older patients. Combining these data with raw fluorescence measurements of RT-qPCR tests for the presence of SARS-CoV-2 genes provides evidence that false-negative rates stem from low viral loads at the single-patient level.

Our study has several limitations. First, we treat all positive tests as true positive. While errors may occur, the rate of false-positive results is very low 2-5 and we do not expect it to significantly affect our results. Future studies can further improve the reliability of confirmation of positive cases by combining PCR test results with serology tests. Second, we treat negative results at the end of test series as 'true-negative', while it is possible that if the test series were continued new positive tests might have been detected. Again, we do not expect this to significantly affect our results: most series in our study end with two consecutive negative results, and the chances for two consecutive false-negative tests are very low. Moreover, this bias will mostly affect the calculated false-negative rate at later days after diagnosis. Third, as viral loads after infection first increase and only later decrease, it is possible that false-negative rates follow an opposite pattern: first decreasing and only later increasing. Analysing our cohort, we could only identify the later phase of increasing false-negative rate. However, it is possible that with different cohorts or inclusion criteria, both phases can be observed. Fourth, it will be interesting to see how changing the way Ct is calculated can fine-tune the way positive and negative results are determined based on conflicting results of the genes. Finally, we emphasize that most of the patients in the study cohort were symptomatic; therefore, our results may not represent the false-negative rate for asymptomatic patients.

Despite these limitations, our results provide important epidemiological and clinical input as to the patient specific sensitivity of tests, with important implications for epidemiological policy for patient-specific quarantine decisions and disease prevention and control. In particular, they underscore that the risk of false-negative at the very early days following diagnosis might be lower than previously thought, reinforcing the use of tests for disease prevention and individual quarantine assessments. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ;  day of diagnosis, were omitted). Patients are sorted by the dates, relative to diagnosis, of their first negative result, then by the relative date of the last positive result. ( C ) Frequency of test results per day relative to diagnosis.

. CC-BY-NC 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; groups (<40 and ≥40, dark and light grey, respectively) calculated separately for early and late days after diagnosis (hatched and empty, respectively). Fisher exact test (Methods: 'Differences in FNR between age groups'). * -p-value<0.05, ** -p-value<0.01. Error bars indicate SD.

. CC-BY-NC 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; https://doi.org/10.1101/2020.10.30.20222935 doi: medRxiv preprint S1A S1B . CC-BY-NC 4.0 International license It is made available under a perpetuity.

is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; https://doi.org/10.1101/2020.10.30.20222935 doi: medRxiv preprint S1C Supplementary Figure 1 . Differential change in Ct value along time after day of diagnosis for different age groups. In the first 4 days after diagnosis, Ct values of E ( A ) and RdRp ( B ) genes are lower for patients ≥40 (light grey) than for patients <40 (dark grey). The opposite is true for the IC gene ( C ) in agreement with within tube competition for reagents between the multiplexed reactions. Error bars indicate SE. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. is the author/funder, who has granted medRxiv a license to display the preprint in (which was not certified by peer review) preprint

The copyright holder for this this version posted November 3, 2020. ; https://doi.org/10.1101/2020.10.30.20222935 doi: medRxiv preprint

COVID-19 Map -Johns Hopkins Coronavirus Resource Center

Specificity and Predictive Values of Molecular and Serological Tests for COVID-19: A Longitudinal Study in Emergency Room

Evaluation of a quantitative RT-PCR assay for the detection of the emerging coronavirus SARS-CoV-2 using a high throughput system

Substantial underestimation of SARS-CoV-2 infection in the United States

Virological assessment of hospitalized patients with COVID-2019

Chest CT for Typical Coronavirus Disease 2019 (COVID-19) Pneumonia: Relationship to Negative RT-PCR Testing

Correlation of Chest CT and RT-PCR Testing for Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases

False-negative RT-PCR in SARS-CoV-2 disease: experience from an Italian COVID-19 unit

Evaluating the accuracy of different respiratory specimens in the laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections

SARS-CoV-2 turned positive in a discharged patient with COVID-19 arouses concern regarding the present standards for discharge

False Negative Tests for SARS-CoV-2 Infection-Challenges and Implications

Diagnostic Performance of CT and Reverse Transcriptase Polymerase Chain Reaction for Coronavirus Disease 2019: A Meta-Analysis

Sensitivity of RT-PCR testing of upper respiratory tract samples for SARS-CoV-2 in hospitalised patients: a retrospective cohort study

Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure

Estimating false-negative detection rate of SARS-CoV-2 by RT-PCR

Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study

Temporal dynamics in viral shedding and transmissibility of COVID-19

Pediatric SARS-CoV-2: Clinical Presentation, Infectivity, and Immune Responses

An analysis of SARS-CoV-2 viral load by patient age

Comparison of viral levels in individuals with or without symptoms at time of COVID-19 testing among 32,480 residents and staff of nursing homes and assisted living facilities in Massachusetts

SARS-CoV-2 detection, viral load and infectivity over the course of an infection

Association of viral load with serum biomakers among COVID-19 cases

Dynamic profile of RT-PCR findings from 301 COVID-19 patients in Wuhan, China: A descriptive study

FALSE-NEGATIVE RESULTS OF INITIAL RT-PCR ASSAYS FOR COVID-19: A SYSTEMATIC REVIEW

Distinct characteristics of COVID-19 patients with initial rRT-PCR-positive and rRT-PCR-negative results for SARS-CoV-2

Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR

Stability issues of RT-PCR testing of SARS-CoV-2 for hospitalized patients clinically diagnosed with COVID-19

False negative of RT-PCR and prolonged nucleic acid conversion in COVID-19: Rather than recurrence

Viral load of SARS-CoV-2 in clinical samples

SARS-CoV-2 Viral Load in Upper Respiratory Specimens of Infected Patients

SARS-CoV-2 viral load in clinical samples of critically ill patients

Prolonged virus shedding even after seroconversion in a patient with COVID-19

Viral load dynamics and disease severity in patients infected with SARS-CoV-2 in Zhejiang province, China

Predicting infectious SARS-CoV-2 from diagnostic samples

Practical considerations in design of internal amplification controls for diagnostic PCR assays