key: cord-0821203-obn3jp8k authors: Wu, Yingping; Xu, Wei; Zhu, Zhiqiang; Xia, Xiaoping title: Laboratory verification of an RT‐PCR assay for SARS‐CoV‐2 date: 2020-08-04 journal: J Clin Lab Anal DOI: 10.1002/jcla.23507 sha: 8f65ccea0323c8f0c3879ce986f62c3c5bf31d59 doc_id: 821203 cord_uid: obn3jp8k BACKGROUND: Reverse transcription‐polymerase chain reaction (RT‐PCR) is an extremely common clinical method for detecting pathogens, particularly for emerging infectious diseases such as the new coronavirus disease (COVID‐19). Currently, detection of the RNA from the novel coronavirus SARS‐CoV‐2 is the gold standard for establishing a COVID‐19 diagnosis. This study evaluates the characteristic performance of the analytical system in a clinical laboratory. METHODS: A commercial SARS‐CoV‐2 RNA RT‐PCR Kit used in a clinical laboratory is assessed based on ISO 15189 verification requirements. A multiple real‐time RT‐PCR assay for the RdRP, N, and E genes in SARS‐CoV‐2 is verified. RESULTS: The analytical system exhibits good analytical sensitivity (1000 copies/mL) and specificity (100%); however, the values of 86.7% and 100% for analytical accuracy deserved attention, compared with two other types of methods. Overall, the kit is potentially useful for SARS‐CoV‐2 diagnostic testing and meets the verification requirements. CONCLUSION: Compliance with international standards, such as ISO 15189, is valuable for clinical laboratories and for improving laboratory medicine quality and safety. Normalization is essential for obtaining reliable results from the SARS‐CoV‐2 RNA RT‐PCR assay. This study aims to develop an improved SARS‐CoV‐2 verification framework compared with traditional molecular diagnostic methods, given the urgency of implementing new assays in clinical laboratories. RNA was extracted from samples arranged in pairs using a Viral RNA Extraction Kit and the Ex2400 extraction system (Liferiver). A sample volume of 300 µL was used for RNA extraction, and the elution volume was 50 µL. RT-PCR was performed using a Novel Coronavirus Real-Time RT-PCR Kit (Liferiver) according to the manufacturer's protocol. Primers and probes were designed to target SARS-CoV-2 (GenBank accession number: MN908947). The following one-step PCR protocol was used: one cycle at 45°C for 10 minutes and 95°C for 3 minutes, followed by 45 cycles at 95°C for 15 seconds and 58°C for 30 seconds, with single-point fluorescence detection at 58°C. Finally, 25 µL of the total PCR volume was used according to the manual protocol. The detection limit of the RdRP qRT-PCR assays was approximately 1.0 × 10 3 copies/mL. Crossing point (Cp) values were used to determine SARS-CoV-2. The positive predictive value (PPV) and negative predictive value (NPV) indicate the analytical accuracy. At least 5 negative and positive samples (which should include weak positive/low-amplification samples), and generally no fewer than 10 samples, should be selected for the final calculation. In accordance with the patient sample detection procedure, two reference methods and a candidate method are used for parallel detection. The assessment criterion is the performance declared by the manufacturer of the kit. Both reference kits were officially recommended by registration with the National Medical Products Administration. Analytical specificity in terms of cross-reactivity is determined by With reference to the CNAS-GL039:2019 guidelines, certified reference materials declared that the limit of detection (LOD) concentration should be diluted. In this study, a low-level sample with a viral load of 1.0 × 10 3 copies/mL was used. The assessment criteria of the LOD test required detection in more than 90% of the samples (18 of the 20 positive samples). The mean value and standard deviation (SD) were calculated with reference to the Cp values of the RdRP/ N/E genes, and viral loads were calculated by the standard curve of the RdRP gene. Validated examination procedures were subject to independent verification in our laboratory. Analytical accuracy, sensitivity, and specificity are the essential verification parameters in qualitative testing; repeatability and stability are optional parameters. However, we evaluated all performance parameters of the SARS-CoV-2 assay in this study (Figure 1 ). To evaluate accuracy, 30 positive samples from patients with a respiratory disease (20 positive and 10 negative samples were tested with other RT-PCR reagents) were tested in our analytical system. The results were consistent with those derived using reference method (a), with a positive predictive value (PPV) of 100% and a negative predictive value (NPV) of 100%. However, 83.3% (62.6%~95.3%, 95% confidence interval (CI)) of PPV and 100% of NPV (54.1%~100%, 95% CI) were calculated for reference method (b), which showed 86.7% (69.3%~96.2%, 95% CI) accuracy. The two reference methods have different analytical sensitivities of 1000 copies/mL and 500 copies/ mL, and the kit that we used is more similar to reference method (a) ( Figure 1 ). We evaluated the low-amplification control for sensitivity testing to calculate the LOD of the analytical system, which is defined as the lowest concentration (1 × 10 3 copies/mL for viral load) at which 100% of positive samples are detected. Viral RNA in 20 samples (the logarithmic value of viral load for the RdRP gene was 3.27 ± 0.30 (mean ± SD)) was detected with a positive rate of 100% in this study ( The analytical specificity of the SARS-CoV-2 RT-PCR assay was eval- Mucin was added to two levels of positive samples until a final concentration of 1 mg/mL was achieved. Based on the interference results, 1 mg/mL mucin did not exert a significant effect on the RT-PCR assay ( Figure 2 ). Most kits for SARS-CoV-2 RNA testing have limitations, such as a brief development time, insufficient testing in preclinical trials, sensitivity is often referred to as the LOD, and the lowest actual concentration in a specimen should be detected. 7 Relevant validation and verification reports evaluating the performance of the SARS-CoV-2 analytical system in public use are not available. We consider this paper to be the first one to focus on improving internal quality control. This study was performed in a SARS-CoV-2 RNA testing laboratory that acquired ISO 15189 accreditation from the CNAS three years ago. From 29 January 2020 to 29 February 2020, 1759 tests for the SARS-CoV-2 RNA were completed in our laboratory. Currently, SARS-CoV-2 is spreading relatively rapidly throughout China and worldwide. Clinical laboratories must improve the quality management of SARS-CoV-2 RNA tests, particularly in some laboratories with poor operational conditions in developing countries. Before a new test is used in a clinical laboratory, the performance characteristics of the procedure must be confirmed. 9 ISO 15189 defines the term "verification" as "confirmation through the provision of objective evidence, that specified requirements have been fulfilled." Therefore, analytical verification of qualitative tests is absolutely evidenced by the accuracy, sensitivity, specificity, and interference data. [10] [11] [12] The Real-Time Multiplex RT-PCR Kit (detection of 3 genes) can be performed basically consistently with the manufacturer's statement. Note: Repeatability precision was determined by testing the limit of detection in a sample with a viral load of 1000 copies/mL, and results showed 100% detection at the LOD. Abbreviation: NA, undetectable result. The verification performance of the SARS-CoV-2 RNA test satisfied the assessment criteria, with a good result for the LOD, PPV, NPV, cross-reactivity, and interference. However, we must consider sin- and detection areas) when the reagent is established. In our study, differences were found in the sensitivity of the three target genes, and the E gene has a 20% (2/20) missed detection rate (Table 1) . Overall, reagent manufacturers should establish reagent methods based on certain scientific and experimental data. Both the PPV and NPV were 100% against the reference method (a), which has the same LOD concentration of 1000 copies/mL. To our surprise, a PPV of 83.3% and an NPV of 100% were obtained for reference reagent (b), which has higher sensitivity with an LOD concentration of 500 copies/mL. In our study, this kit for SARS-CoV-2 RNA detection may not produce results consistent with the manufacturer's statement of 100% accuracy. Therefore, verification of the accuracy with other reference methods is needed in future studies. The LOD is another important performance characteristic of both quantitative and qualitative tests. Analytical performance at the low concentration limit is often defined as the ability of the test to diagnose a disease. The manual provided with the kit indicated that a concentration of 1000 copies/mL with a Cp value of 43 was considered the minimal concentration. At present, no international or national standard substance of SARS-CoV-2 RNA is available; however, reference materials of SARS-CoV-2 RNA can be acquired, and the quantitative value can be determined by ddPCR. The reference material containing 2.0 × 10 3 copies/mL was diluted to 1.0 × 10 3 copies/mL as low-level positive samples for testing (Table 1) without retesting? Accordingly, we suggest that reagent manufacturers need more data to optimize reagent performance. In summary, verification of the SARS-CoV-2 RNA test was consistent with the product requirements, and the detection system basically meets the detection performance stated in the kit. Finally, we postulate that this study will be useful for other clinical laboratories, before using a new analytical system for molecular diagnosis. All authors confirm that they contributed to the content of this paper. W Xu and ZQ Zhu assisted with sample testing. YP Wu and XP Xia were responsible for the study design and data interpretation and were major contributors to manuscript editing and critical revision of the article. All authors read and approved the final manuscript. We are grateful to all colleagues in our laboratory for their support. Yingping Wu https://orcid.org/0000-0002-3456-7705 A SARS-like cluster of circulating bat coronaviruses shows potential for human emergence Origin and evolution of pathogenic coronaviruses Clinical features of patients infected with 2019 novel coronavirus in Wuhan Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study Novel Coronavirus (COVID-19) The role of laboratory diagnostics in emerging viral infections: the example of the Middle East respiratory syndrome epidemic Validation of laboratory-developed molecular assays for infectious diseases On the origin and continuing evolution of SARS-CoV-2 Medical laboratories-Requirements for quality and competence Guidance on the Application of Accreditation Criteria for the Medical Laboratory Quality and Competence in the Field of Molecular Diagnostics Guidance on the Performance Verification for Molecular Diagnostic Procedures. China National Accreditation Service for Conformity Assessment Tezak-Fragale Z. MM03-A3: Molecular Diagnostic Methods for Infectious Diseases