key: cord-0947843-1q1gydrw
authors: Mueller, L.; Scherz, V.; Greub, G.; Jaton, K.; Opota, O.
title: Computer-aided medical microbiology monitoring tool: a strategy to adapt to the SARS-CoV-2 epidemic and that highlights RT-PCR consistency
date: 2020-07-29
journal: nan
DOI: 10.1101/2020.07.27.20162123
sha: 31baefd2a992342a9db115ca64ff6f165db9e8bd
doc_id: 947843
cord_uid: 1q1gydrw

Since the beginning of the COVID-19 pandemic, important health and regulatory decisions relied on the SARS-CoV-2 reverse transcription polymerase chain reaction (RT-PCR) results. Our diagnostic laboratory faced a rapid increase in the number of SARS-CoV-2 RT-PCR, with up to 1,007 tests per day. To maintain a rapid turnaround time to support patient management and public health authorities' decisions, we moved from a case-by-case validation of RT-PCR to an automated validation and immediate transmission of the results to clinicians. To maintain high quality and to track possible aberrant results, we developed a quality-monitoring tool based on a homemade algorithm coded in R. We present the results of this quality-monitoring tool applied to 35,137 RT-PCR results corresponding to 30,198 patients. Patients tested several times led to 4,939 pairwise comparisons; 88% concordant and 12% discrepant. Among the 573 discrepancies, 428 were automatically solved by the algorithm. The most likely explanation for these 573 discrepancies was related for 44.9% of the situations to "Clinical evolution", 27.9% to "Preanalytical" problems, and 25.3% to "Stochastic". Finally, 11 discrepant results could not be explained, including 8 received from external partners for which clinical data were not available. The implemented quality-monitoring strategy allowed to: i) assist the investigation of discrepant results ii) focus the attention of medical microbiologists onto results requiring a specific expertise and iii) maintain an acceptable TAT. This work highlighted the high RT-PCR consistency for the detection of SARS-CoV-2 and the importance of automated processes to handle a huge number of samples while preserving quality.

The rapid spread and the high incidence of COVID-19 pandemic caused 48 unprecedented challenges for diagnostic microbiology laboratories. Rapid and high 49 throughput SARS-CoV-2 reverse transcription polymerase chain reaction (RT-PCR) 50 have been quickly developed as the ones proposed by . These 51 molecular diagnostic assays rapidly became the cornerstone of patient diagnosis as 52 well as hospital and public health managements. Consequently, microbiology 53 laboratories were reorganized to respond to the high demand for SARS-CoV-2 testing 54 (4). This situation required (i) the rapid adaptation of infrastructures, (ii) quick validation The limited experience on these newly implemented RT-PCR assays, including their 76 performance (7), highlighted the need for an active surveillance of the quality of 77 provided results. Delta checks are commonly used in clinical chemistry laboratories to 78 monitor analytical throughputs that outreach capacity for sample-by-sample validation. 79 Delta checks describes a process where discrepancies in sequential results of the 80 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020 . . https://doi.org/10.1101 same patient are detected to prompt repetition of the analysis (8). We wondered 81 whether a similar approach could be used to monitor the quality of SARS-CoV-2 82 results obtained in our laboratory. However, Delta checks are usually restricted to 83 analytes exhibiting limited short-term variations and is as such unsuitable to 84 microbiology results. Thus, we developed a quality monitoring methodology based on 85 a homemade algorithm programmed in R to monitor SARS-CoV-2 RT-PCR results. 86 The developed quality monitoring methodology leveraged repeated testing to identify 87 potential preanalytical or analytical culprits as well as cases requiring further 88 biomedical investigations. The algorithm developed in-house accounts for the 89 expended variability among results, to restrict the list of discrepancies to cases truly 90 requiring investigation.

In this article, we present the results obtained from the application of our quality 93 surveillance on data from the first four months of SARS-CoV-2 crisis in our laboratory.

Beside its role as quality management tool, application of this surveillance allowed us 95 to quickly gain knowledge about this RT-PCR applied to a novel virus and new 96 disease. In particular, this process allowed us to identify clinical specimen with 97 significant added value and the presence of long-term carrying patients. Samples collected from patients with a suspected COVID-19 or for screening were 102 tested by RT-PCR, using either our high-throughput MDx platform (5), the cobas 103 SARS-CoV-2 qualitative test (Roche, Basel, Switzerland) and the Xpert SARS-CoV-2 104 test (Cepheid, California, USA). The E gene was targeted by the RT-PCR performed 105 on the MDx platform (5), as described by Corman and colleagues (1). The cobas 106 SARS-CoV-2 targeted the E gene as well as the ORF1/a and was performed 107 according to the manufacturer guidelines. Finally, the Xpert SARS-CoV-2 test targets 108 the N gene and the E gene. The three methods displayed similar performances for the 109 detection of SARS-CoV-2 from various clinical specimens and similar cycle threshold 110 (Ct) value when positive (9-11). Samples were mainly collected from the upper 111 respiratory tract. However, other type of samples were also tested (Table S1) .

Nasopharyngeal and oropharyngeal swabs were collected in Copan Universal 113 Transport Medium System (UTM-RT) or BD™ Universal Viral Transport System 114 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 29, 2020 . . https://doi.org/10.1101 (UVT). Sputa and bronchial aspirates were liquefied using N-acetyl-L-cysteine prior 115 analysis using the cobas 6800 system, or prior nucleic acid extraction using the 116 MagNAPure 96 instrument when samples are tested on our MDx Platform (12, 13).

Anorectal swabs were collected as previously described (14).  "Low yield" described samples type rarely or never observed as positive (i.e. 143 less than 5% of times, Table S1 ). Thus, the algorithm classified as "Low yield" 144 discrepancies where a low yield sample returned a negative result (explainable 145 by sample nature, e.g. blood and urine).

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint  "Time delay" explanation was retained by the algorithm when the time interval 147 between the two compared samples was over 10 days. Indeed, in this case, the 148 discrepancy would be explained by the evolution of the disease (new infection 149 or disease resolution).

 "To be investigated" was finally retained by the algorithm for discrepancies 151 meeting none of these criteria and requiring further investigation by medical 152 microbiologists.

The script was designed to compare each sample only to the last relevant result. For 154 instance, a positive nasopharyngeal swab followed by a negative PCR in blood, and 155 then later by another positive nasopharyngeal swab, will lead to only one discrepancy: 156 the negative blood classified as a "Low yield". The second nasopharyngeal swab will 157 be classified as concordant with the previous nasopharyngeal swab. Of note, two 158 nasopharyngeal samples taken more than 10 days apart once negative and once 159 positive with a Ct > 35, would be classified as "Stochastic" and not as "Time delay".

Indeed, discrepancies were classified according to the first matching criteria in the 161 following order: "Low yield", "Stochastic", "Time delay" and "To be investigated", as (Table S3 ). In this manual analysis, discrepancies between two 170 nasopharyngeal swabs taken within the same period (< 24h) and collected in different CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint corresponding testing phases or context: clinical context, preanalytical or stochastic 180 (Tables S2 and S3) . 

Our pipeline significantly reduced the number of discrepancies requiring human investigation 210 and a probable explanation could be identified for most of the discrepant results. Indeed, the 211 majority of the discrepant pairwise comparisons (n=428/573) could be automatically attributed 212

by the pipeline to a putative explanation ("stochastic", "low yield" or "time delay") ( Fig. 3A and 213 3B, Table S2 ). Only the remaining discrepancies (n=145) did not fit any of the solving rules 214 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint encoded in the algorithm and required investigations based on the available analytical and 215 clinical information (Table S3) . 216

The profiles of putative explanations for discrepancies evolved depending on the time interval 218 between the compared analyses (Fig. 3C) . In samples received among the same day, our 219 assessment explained 77.3% (n=34/44) of the discrepancies as related to the preanalytical 220 phase (i.e. explained by the sample types or sample collection in different health centers), 221 followed by stochasticity (Ct value >35) at 22.7% (n=10/44). The discrepancies in results of 222 samples received 1-3 days apart were explained by factors affecting the preanalytical phase 223 at 55.4% (n=51/92), followed stochasticity at 32.6% (n=30/92). Interestingly, 7 of the 224 discrepancies observed in the 1-3 days interval were explainable by nosocomial (n=6) or 225 community (n=1) acquired infections based on health records, which could explain the quick 226 negative to positive transition. These 7 discrepancies were thus classified in the clinical 227 evolution context. As for the 4 remaining discrepancies, clinical records were not available for 228 3 and the last one remained unexplained. Investigations of discrepancies between samples 229 received 4-10 days apart incriminated the preanalytical phase at 41.6% (n=55/132), followed 230 by stochasticity at 30.3% (n=40/132). As expected, the discrepancies imputable to the clinical 231 evolution of the disease based on clinical records (new infection or infection resolution) was 232 greater in the 4-10 days interval since it represented 22.7% of the discrepancies (n=30/132). 233

The 7 remaining discrepancies in this time interval could not be explained, either in absence 234 (n=5) or in presence (n=2) of clinical information. Over 10 day, the clinical evolution of the 235 disease was the main explanation (72.1%, n=220/305) for discrepancies, as it was the default 236 explanation retained by our automatic pipeline for discrepant results from samples collected 237 more than 10 days apart in absence of any other explanation. 238

In the overall assessment of the 573 discrepancies from samples taken 90 days apart, 44.9% 240 (n=257) of discrepancies could be explained by the clinical evolution of the disease (e.g. for discrepancies could be found for 3 cases, despite the availability of full clinical 251 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint documentation. Moreover, short-term negative to positive transitions were compatible with 21 252 nosocomial and 8 community-acquired infections based on clinical records (Table S3) . 253 254 Evolution of the discrepancy patterns across the epidemic period 255

The pattern in transitions (negative result followed by a positive result or the reverse) among 256 discrepancies evolved over the studied period of four months. In the first two months included 257 in the analysis, which corresponds to the first two months of the epidemic in our region (12.02-258 12.04.2020), 71.3% of negative to positive transitions were observed (n=154/216). CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10. 1101 An evaluation of the performance of the RT-PCR for SARS-CoV-2 detections based on 288 samples collected over a short time interval showed a good level of negative and positive 289 agreements. Indeed, an initial negative result on an URT sample was confirmed in 99.2% 290 (243/245) of cases for patients tested twice on the same day (Table 1) RT-PCRs performed in our diagnostic laboratory from February 12 to June 12, 2020, the 305 algorithm identified 3,214 patients owing multiple tests. These patients represented an 306 opportunity for quality assessment of our analyses, but also required careful attention to 307 investigate potential discrepancies. Among the 3,124 patients tested multiple times, we The natural evolution of the disease appeared as the main origin for discrepancies among 320 repeated testing in our classification. Indeed the clinical context contributed to explain almost 321 half (44.9%) of the discrepancies assessed by automatic resolution and manual investigation 322 together. The positive to negative transition, compatible with a solved infection, was the most 323 frequent pattern (61%). This is consistent with the studied period, which integrated the 324 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint development of the epidemic in our region and the transition phase where fewer cases 325 occurred and the infection resolved in a majority of patients. The automatic algorithm classified 326 all discrepancies between samples collected more than 10 days apart as related to the clinical 327 evolution of the disease, in absence of any other putative explanation. This rule was designed 328 to limit the need for manual review of discrepant cases that were not expected to be 329

informative, but could have biased discrepancies classification toward this category. 330 331 Preanalytical factors were incriminated in 27.9% of the observed discrepancies. Negative 332 result from samples types rarely or never positive, such as blood and urine, was the most 333 frequent explanation retained for these discrepancies (47.5%) (Table S2 ). Among the 334 remaining 52.5% of discrepancies associated to preanalytical phase, 25.0% were due to the 335 comparisons of two samples of different nature (i.e. an URT sample and a rectal swab). Both 336 of these specimens presented over 5% of positivity in our dataset and thus were considered 337 as samples with an interesting clinical yield. Yet, positivity in only one of the sampling site has 338 already been reported (19). Thus, comparisons of distinct sample collection sites was 339 considered in our assessment as a sufficient explanation for the observed discrepancy. The 340 remaining 27.5% of the discrepancies, occurred after repeated collection of samples in less 341 than 10 days but in different healthcare provider (Table S2) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020 . . https://doi.org/10.1101 agreements of RT-PCR results among URT samples collected the same day were of 88.9% 362 and 99.2% respectively. The positive concordance rate exhibited an important drop when 363 samples were collected more than 1 day apart, with the lowest concordance agreement 364 (33.3%) observed in a 4-6 days time delay. This might be explained by the clinical evolution 365 of patients. Nevertheless, these positive agreement rates should be considered with caution, 366

due to the small number of positive samples for which additional testing were requested (i.e. 367 in a time interval of 4-6 days 468 negative samples were retested, while for positive samples 368 only 21 were reanalyzed). In fact, patients were 4x times more often retested 1-72 hours after 369 a first initial negative results (2.5%) than after a first positive result (0.6%) offering more 370 opportunities to assess negative than positive agreement. Another limitation of our work is that while we intended to use an unbiased algorithm stable 387 over time to investigate discrepancies in results, some of the applied criteria were partly 388 arbitrary (e.g. the 10 days limit to consider discrepancies as expected due to the clinical 389 evolution of the disease). Furthermore, our process consent the use of a single explanation 390 for each observed discrepancy, while more could be applicable. While arguable, these choices 391 were made to fit a strategy of quality monitoring. Indeed, the primary aim of the presented 392 methodology was to attribute the observed discrepancies to the most likely explanation to 393 focus on truly unexplainable and problematic cases. 394

Clinical laboratory vulnerabilities during the COVID-19 pandemic were the subject of a recent 396 publication by Lippi et al. (26) . Our assessment overlaps with some of the preanalytical culprits 397 identified by the authors such as specimen collection (see "detailed explanations" Tables S2  398   . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint and S3). However, some other potential vulnerabilities were not considered as probable 399 causes for discrepancies in our assessment, since they are covered by other pre-existing 400 quality management procedures in our laboratory. For instance, our preanalytical team 401 systematically rejects samples missing a patient identification. Moreover, in our laboratory, 402 internal extraction controls and amplification controls are systematically included to detect 403 samples that might contain interfering substances compromising the amplification (9). repeatedly in a short timeframe showed consistent results, displaying the good reproducibility 435 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020 . . https://doi.org/10.1101 of the RT-PCR for SARS-CoV-2 detection. Application of this method for quality monitoring 436 enabled to focus on problematic cases requiring biomedical expertise while maintaining an 437 acceptable TAT. 438 439 Acknowledgements. 11. Jacot D, Greub G, Jaton K, Opota K. 2020 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. here. Data are distributed according to their reception date. Blue bars represent samples for 541 which RT-PCR results were negative (88%) while orange bars depict positive samples (12%). 542 543 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint 544 545 Figure 2 . SARS-CoV-2 analytical flowchart. An R-based script was used to identify patients 546 with multiple, potentially discrepant, results (upper part of the chart) and therefore insure their 547

surveillance. The algorithm processed further, discrepant results and a potential explanation 548

for the observed discrepancy was attributed either automatically either manually (lower part of 549 the chart and Fig. S1 ) 550 551

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint 552 Figure 3 . Time laps between discrepant analyses, putative phase assignment and 553 cumulative score. 573 pairwise comparisons are represented according to their date of 554 reception and colored depending on the analytical phase which best explained the observed 555 discrepancy. A and B. Before manual curation discrepancies were classified by the algorithm 556 as "Preanalytical", "Stochastic", "Clinical evolution" and "To be investigated". C and D. After 557 manual curation comparisons previously categorized as "To be investigated" were re-assigned 558

to the same categories or as "Unsolved". A and C. Time laps between discrepant analyses. 559

Transitions from a negative to a positive result are represented in the positive side of x axe, 560

while positive to negative transitions are plotted on the negative side. A. Before manual 561

investigation. Only comparisons of samples taken less than 10 days apart required 562 investigation since the "Clinical evolution" was otherwise assigned by default by the algorithm. 563 C. After manual investigation. Short term discrepancies were mostly attributed to 564 "Preanalytical" factors and "Stochastic" phenomenon. Discrepant analyses putatively 565 explained by "Clinical evolution" of the disease was naturally retained for samples collected 566

within larger time interval. B and D. Cumulative score of discrepant analyses. B. 74.7% of 567 comparisons were solved and classified, while 25.3% were classified as "To be solved" and 568 needed further investigations. D. In a 90 days interval cumulative score shows that most 569 discrepancies could be explained by the "Clinical evolution" of the disease. Details on 570 discrepancies putative classification and phases are detailed in Materials and Methods 571 section. 572

Discrepant pairwise comparisons Discrepant pairwise comparisons . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint SARS-CoV-2 detection (n=250). 577 578 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10.1101/2020.07.27.20162123 doi: medRxiv preprint This histogram represents the Ct value (maximal if more than 1) for SARS-CoV-2 detection 581 by RT-PCR in the last positive sample of patients with sustained positivity in samples taken 582 30 days apart or more. 583 584 . CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10. 1101 Tables 585 Table 1 

. CC-BY-NC-ND 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review)

The copyright holder for this preprint this version posted July 29, 2020. . https://doi.org/10. 1101 

Detection of 2019 novel coronavirus 450 (2019-nCoV) by real-time RT-PCR

SARS-CoV-2, un point dans 453 la tourmente

455 Diagnostic strategies for SARS-CoV-2 infection and interpretation of microbiological 456 results

Clinical microbiology laboratory adaptation to COVID-19 459 emergency: experience at a large teaching hospital in

Ten years of R&D anf full automation 462 in molecular diagnosis

Laboratory Turnaround Time

Performance du frottis 465 nasopharyngé-PCR pour le diagnostic du Covid-19 Recommandations pratiques sur 466 la base des premières données scientifiques

Delta Check Practices and Outcomes: A 468 Q-Probes Study Involving 49 Health Care Facilities and 6541 Delta Check Alerts

Added 484 Value of Xpert MTB/RIF Ultra for Diagnosis of Pulmonary Tuberculosis in a Low-485 Prevalence Setting

High prevalence of anorectal 488 chlamydial infection in HIV-infected men who have sex with men in Switzerland

Value of Diagnostic Testing for SARS-CoV-2/COVID-19

Comparison of 501 nasopharyngeal and oropharyngeal swabs for SARS-CoV-2 detection in 353 patients 502 received tests with both specimens simultaneously

Positive Sputum and Feces After Conversion of Pharyngeal Samples in Patients With 505 COVID-19

Factors associated with 507 duration of viral shedding in adults with COVID-19 outside of Wuhan, China: A 508 retrospective cohort study

Evaluating the accuracy of different respiratory specimens in the laboratory 512 diagnosis and monitoring the viral shedding of 2019-nCoV infections

Errors in clinical laboratories or errors in laboratory medicine?

517 Preanalytical errors in medical laboratories: a review of the available methodologies of 518 data collection and analysis

Quantitative Detection and Viral Load Analysis 521 of SARS-CoV-2 in Infected Patients

Sensitivity of RT-PCR testing of upper respiratory tract samples 525 for SARS-CoV-2 in hospitalised patients: a retrospective cohort study

The critical role of laboratory medicine during coronavirus 528 disease 2019 (COVID-19) and other viral outbreaks

Internal quality assurance in diagnostic 531 microbiology: A simple approach for insightful data

Medical errors: impact on clinical laboratories and other critical areas