key: cord-0719284-yymqf4ex authors: Li, Rong-Hai; Wang, Qing-Yong title: A localized small-scale external quality assessment (EQA) for PCR testing of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in the molecular laboratories date: 2021-12-23 journal: J Virol Methods DOI: 10.1016/j.jviromet.2021.114441 sha: e9250a779869e31dfc4bc47ee4041429427c8eef doc_id: 719284 cord_uid: yymqf4ex The aim of this study was to estimate the PCR results for SARS-CoV-2 testing in 32 participating laboratories in a localized small-scale external quality assessment (EQA) scheme. EQA samples were distributed to the participants and detected immediately on the day of delivery. Qualitative results were submitted to the EQA provider, including negative or positive results along with cycle threshold (Ct) values for different target genes. Although the variability of Ct values differed among the laboratories in the EQA, a total of 32 (100%) participants reported correct qualitative results. The study showed that the mean loads of N or E gene were higher than those of ORF1ab in SARS-CoV-2 RNA samples. Regardless of the analyzed gene target, the mean Ct values for weak positive and positive samples varied by fewer than 1.74 and 1.91 cycles, respectively. Less than 12% of reported Ct values for ORF1ab and N genes deviated by more than ±4 (maximum −9.92) cycles, while none deviated by more than ±4 cycles for the E gene. The current EQA program can provide a robust practical basis for follow-up planning to conduct evaluations for SARS-CoV-2 PCR testing and other novel emerging pathogens in the future. Robust, accurate, and reproducible molecular testing is essential to control the novel coronavirus disease , needing the cooperation of public health agencies, clinical laboratories, and industry [1] . Viral nucleic acid testing has played an important role in control of the new disease, being regarded as the diagnostic gold standard. Many commercial kits detecting SARS-CoV-2 continue to come into the market, and laboratory workers must perform verification testing before carrying out testing of clinical samples. Quality management and improvement within the laboratories require urgent assistance from external quality assessment (EQA), especially for newly established molecular laboratories. Testing of standardized EQA samples could assess the laboratories' abilities and identify inaccurate or suboptimal combinations of reagents and instruments that may be used by some laboratories. Moreover, false-positive and false-negative results should be analyzed thoroughly. This approach could offer valuable practical suggestions to some EQA panel members for increasing assay reliability and improving their capabilities. Five samples (500 μl each) provided to each enrolled laboratory in the EQA program included one SARS-CoV-2-negative sample (containing mixed MERS and SARS [1×10 4 copies/ml, each]), one SARS-CoV-2-negative sample (negative sample from healthy individual), and three random samples ranging from weak positive (1.61×10 3 copies/ml) to positive (1.45×10 4 copies/ml). All samples were customized: virus-like particles (VLPs) were diluted with phosphate buffered saline at suitable dilution ratios, then were subsequently used for RNA extraction and amplification. cDNA copies were quantified by a digital PCR system for J o u r n a l P r e -p r o o f 4 deducing the initial VLP concentration. All EQA samples were distributed to each participating molecular laboratory via dry ice delivery on the scheduled day. We recommended that the EQA samples be handled according to the laboratory biosafety interim guidance related to SARS-CoV-2 issued by the World Health Organization [2] . Recommended volumes for RNA extraction and input for PCR testing were 200 μl and 5 μl, respectively. Requested EQA data included RNA extraction/amplification platforms, routine PCR reagents, negative or positive results, and raw Ct values for different gene targets (ORF1ab, N, and E gene), which were submitted online to the EQA provider on the date of sample delivery. Each laboratory could retest the remaining EQA sample volume (stored at −20°C) on the same day using alternative platforms and reagents, but those data were not required to be returned in this EQA program. Table 1 , 32 laboratories were enrolled in the EQA program, utilizing diverse test systems and reagents comprising 18 different extraction instruments, eight different amplification instruments, and nine different PCR amplification reagents. In this study, all participating laboratories detected SARS-CoV-2 by real-time PCR. According to the returned information, all enrolled laboratories followed the advised extraction and input volumes for analysis. All the EQA samples containing SARS-CoV-2 RNA were successfully detected for the tested target genes, while all SARS-CoV-2-negative EQA samples were correctly identified as negative. As shown in Table 2 when Ct values deviate dramatically from the mean Ct value. SARS-CoV-2 PCR testing is an important element for reducing transmission of this pathogen. Molecular testing with maximum accuracy is essential for all PCR laboratories and EQA represents a necessary procedure for ensuring reliable PCR results across all molecular laboratories involved in clinical testing. Although not being obtained from inactivated positive samples of confirmed cases, the SARS-CoV-2 RNA loads of EQA positive samples in the current EQA were far lower than the mean viral loads of most infected individuals [3] . The assessment panel was well-timed to support the newly established molecular laboratories carrying out SARS-CoV-2 PCR testing, because these facilities were not included in the early large-scale EQA activity organized by the National Center for Clinical Laboratories [4] . The EQA assessment was conducted on the scheduled date, including logistics service, PCR testing and result reporting. On the basis of previous EQA data, we found that false-positive or false-negative results were a potential problem [4] [5] [6] [7] . However, the specificity and sensitivity were simultaneously 100% for all the participating laboratories in the current EQA program; the main reasons for which perhaps included the issuing of updated related laboratory operation guidelines by authorities and strict standardized permission for laboratories performing SARS-CoV-2 PCR testing in the capital. Although no laboratory in this EQA reported incorrect results, it remains necessary to keep a close watch on the possibility of contamination derived from Standardization is essential for laboratories to report comparable Ct values, because of a call for diagnostic tests with viral loads in the reports [9] . Consistent with previously reported results, we found that the sensitivity of ORF1ab detection was lower than that of N or E genes in our EQA samples [3] . The laboratories participating in this EQA used a variety of extraction platforms, amplification platforms, and PCR reagents. Thus, Ct values obtained in different laboratories via different methods are challenging to compare. In agreement with other EQA results, the mean Ct values for the positive samples varied by fewer than 1.91 cycles. However, superior to other EQA results, ≥47.1% of the results deviated from the mean value by < ±2 cycles and ≤11.8% deviated by > ±4 cycles [6] . We also found that some laboratories returned positive results along with maximum Ct value deviation from the corresponding mean Ct value, eg ORF1ab (-6.91) and N (-9.92) from weak positive samples, and ORF1ab (6.94) and N (-7.14) from positive samples. These data could provide an important warning for the EQA provider to further monitor specific laboratories. Laboratories should take into consideration concerns related to equipment calibration, pipette calibration, This study had several limitations. First, point-of-care testing platforms were not included in the current EQA program. Second, information regarding alternative platforms and reagents in each laboratory were not collected and analyzed in the current EQA program. In conclusion, this study summarizes an EQA for SARS-CoV-2 PCR testing including 32 public health organization laboratories in Beijing. Some laboratories should be focused on in a follow-up EQA owing to their much higher or lower Ct values compared with mean Ct values for corresponding gene targets. Molecular laboratories should perform verification experiments before carrying out routine PCR tests and monitor the laboratory output by meticulous quality management. Local small-scale EQAs can be useful for molecular laboratories involved in SARS-CoV-2 PCR testing, particularly those with little previous experience, to increase the quality of results and identify potential weaknesses. [9] Service RF. A call for diagnostic tests to report viral load. Science. 2020;370:22. Tables Table 1. Diverse test system/reagent combinations from 32 laboratories enrolled in the EQA. Data are presented as (n) or n (%). Emergence of a Novel Coronavirus Disease (COVID-19) and the Importance of Diagnostic Testing: Why Partnership between Clinical Laboratories, Public Health Agencies, and Industry Is Essential to Control the Outbreak Laboratory biosafety guidance related to coronavirus disease (COVID-19): interim guidance Viral load of SARS-CoV-2 in clinical samples External Quality Assessment for Molecular Detection of Severe Acute Respiratory Syndrome Coronavirus Variable sensitivity in molecular detection of SARS-CoV-2 in European Expert Laboratories: External Quality Assessment Nationwide External Quality Assessment of SARS-CoV-2 Molecular Testing Variability of cycle threshold values in an external quality assessment scheme for detection of the SARS ABT: Beijing Applied Biological Technologies Co., Ltd; NG: Beijing NaGene Diagnosis Reagent Co., Ltd HLKY: Beijing HLKY Co. Ltd; ABI: Applied Biosystems Co. Ltd; SHFX: Shanghai FX Co. Ltd; BND: BND Technologies Co We thank Professor Zhi-Sui Chang and her team for organizing the small-scale EQA program.