key: cord-0712203-c60w1r0r
authors: Sohni, Youvraj
title: Variation in LOD Across SARS-CoV-2 Assay Systems: Need for Standardization
date: 2020-12-09
journal: Lab Med
DOI: 10.1093/labmed/lmaa103
sha: bbfbaff2873c36ab550981700bdf7f2ed2f6b97f
doc_id: 712203
cord_uid: c60w1r0r

Multiple SARS-CoV-2 emergency use authorization (EUA) tests are being used for clinical testing across various clinical testing laboratories for meeting the diagnostic challenges of the ongoing pandemic. However, cross-assay variations in performance characteristics need to be recognized. A better understanding is needed of the clinical implications of cross-assay variation in performance characteristics, particularly in the limit of detection (LOD) of the SARS-CoV-2 assays used for clinical testing. Herein, a snapshot of the diversity of SARS-CoV-2 EUA analytical assay systems including methodologies, assay designs, and technology platforms is presented. Factors affecting the variations in LOD are discussed. Potential measures that may standardize across the various assay systems are suggested. Development of international standards and reference materials for the establishment of performance characteristics may substantially alleviate potential clinical decision-making challenges. Finally, cross-assay variation in LODs among the diverse SARS-CoV-2 diagnostic assays impacts clinical decision-making with multiple assay systems in use and lack of standardization across platforms. International standards in parallel with continued cross-platform studies and collaborative efforts across pertinent healthcare entities will help mitigate some of the clinical decision-making challenges.

various assay systems are suggested. Development of international standards and reference materials for the establishment of performance characteristics may substantially alleviate potential clinical decision-making challenges. Finally, cross-assay variation in LODs among the diverse SARS-CoV-2 diagnostic assays impacts clinical decision-making with multiple assay systems in use and lack of standardization across platforms. International standards in parallel with continued cross-platform studies and collaborative efforts across pertinent healthcare entities will help mitigate some of the clinical decision-making challenges.

Keywords: SARS-CoV-2, RT-PCR, LOD, C T , Standards, COVID-19

In response to the ongoing public health emergency for COVID-19, the U.S. Food & Drug Administration (FDA) has issued emergency use authorization (EUA) for several SARS-CoV-2 diagnostic assays developed by IVD manufacturers, academic centers, and commercial reference laboratories. Several of these EUA assays are now part of the SARS-CoV-2 clinical testing workflow. The list of assays that receive EUA continues to increase, and the complete list of these assays can be accessed from the FDA website. 1 The need for widespread availability of accurate diagnostic testing for SARS-CoV-2 is critical for appropriate and timely clinical decision-making. Multiple testing options are needed as SARS-CoV-2 testing capacity increases across the various clinical testing laboratories. In the foreseeable future, a limited manufacturing allocation of testing kits and related consumables compels individual clinical laboratories to use multiple assay systems to meet SARS-CoV-2 testing needs. 2, 3 It is reasonable to expect that cross-assay variation in performance characteristics of the EUA assays currently in use will have a bearing on the results of patient testing and subsequent clinical decision-making. In the context of the ongoing pandemic, ensuring that patient testing is not compromised is essential. Therefore, the clinical implications of cross-assay variation in performance characteristics must be recognized. This is particularly relevant for those patient specimen testing results that are close to the limit of detection (LOD) of the assay systems in use. These high cycle threshold (C T ) SARS-CoV-2 results may not meet the 95% confidence interval (CI) detection criteria, and such test results may be discordant when comparatively tested across different assay systems. The potential impact on SARS-CoV-2 transmission and the clinical significance of such patient results is yet to be completely understood.

A significant majority of those with SARS-CoV-2 infections present with mild symptoms, are asymptomatic with positive results, or never develop any signs or clinical symptoms of COVID-19. 4 Thus a better understanding of the spectrum of the disease is needed because SARS-CoV-2 infection can be detected before the development of clinical symptoms. Although an ideal objective for a test is to eliminate the possibility of presumed false-negative patient results, the challenge is knowing the clinical implications of SARS-CoV-2 testing results that fall within the window of difference in LOD between the assay systems used in the clinical testing workflow.

Currently the EUA assays in use for SARS-CoV-2 diagnostics are all qualitative tests and generally provide a binary test with a positive or negative result. A third presumptive positive result is part of some tests (for example, the Cepheid GeneXpert assay; Cepheid, Sunnyvale, CA). As per the Centers for Disease Control and Prevention (CDC) recommendations, viral testing for SARS-CoV-2 is considered to be diagnostic when conducted among individuals with symptoms consistent with COVID-19. In addition, diagnostic testing may be considered for asymptomatic individuals with known or suspected recent exposure to SARS-CoV-2 to control transmission or to determine the resolution of infection. Viral testing is considered as screening when conducted among asymptomatic individuals without known or suspected exposure to SARS-CoV-2 for early identification and as surveillance when conducted among asymptomatic individuals to detect transmission hot spots or characterize disease trends. The CDC also recommends testing for all close contacts of persons with SARS-CoV-2 infection and for those in some high-risk settings. 5 Assays with a relatively low LOD may detect SARS-CoV-2 in specimens with low viral loads that may be missed by assays with higher LODs. Viral loads in throat swab and sputum specimens peak at approximately 5-6 days after symptom onset, but specimens taken from an individual with a recently acquired SARS-CoV-2 infection can have low viral loads. One small study suggests that viral load detected in an asymptomatic patient may be similar to that in symptomatic patients, which in turn suggests the transmission potential of asymptomatic or minimally symptomatic patients. 6, 7 Ultimately, correlating virus counts and therapeutic measures with outcomes may result in different strategies of care or isolation.

Viral RNA may be detected from patient specimens weeks after the infectious stage has ended post-onset of symptoms. Sun et al 8 reported persistent shedding of viral RNA in nasopharyngeal (NP) swabs and feces specimens, with the estimated time until loss of RNA detection ranging from 45.6 days for NP swab specimens to 46.3 days for fecal specimens in mild cases of infection and from 48.9 days for NP swab specimens to 49.4 days for fecal specimens in severe cases of infection. However, the authors noted that the median time for throat specimens from mild cases of infection was 15.6 days (95% CI, 11.8-20.7 days) and that the 95th percentile was 32.8 days (95% CI, 25.9-42.3 days). Therefore, Sun et al. 8 suggested that the detection of viral RNA for mild cases of infection in throat swab specimens at the 50th day after illness onset should be a low-probability event, beyond the 95th percentile limit, as for fecal specimens. Similarly, according to van Kampen et al, 9 the probability of detecting infectious virus drops below 5% 15 days post-onset of symptoms. Lan et al 10 reported positive reverse-transcription polymerase chain reaction (RT-PCR) results in throat swab specimens from patients who recovered from mild COVID-19 for 50 days at maximum.

Chen et al 11 studied viral shedding at multiple timepoints in stool specimens and analyzed its correlation with clinical manifestations and the severity of illness. Those authors found that SARS-CoV-2 RNA in stool specimens was not associated with the presence of gastrointestinal symptoms and the severity of illness. A majority of patients may remain positive for viral RNA in the feces after the pharyngeal swabs are returned negative. The results show the presence of SARS-CoV-2 RNA in the feces of patients with COVID-19 and suggest the possibility of SARS-CoV-2 transmission via the fecal-oral route. 9 Gupta et al 12 reported that the duration for fecal shedding of viral RNA ranged from 1 to 33 days and in one patient was up to 47 days from symptom onset, longer after the clearance of respiratory specimens. In spite of viral RNA concentration, virus isolation from stool specimens was not successful from patient specimens collected between days 6 and 12. There appears to be active replication of the virus in the throat during the first 5 days after the onset of symptoms with no virus replication in stool specimens. 13 Overview www.labmedicine.com An RT-PCR assay may have the analytical sensitivity for detecting a few hundred copies of viral RNA, but that does not indicate the infectious status of the patient. For these reasons, identifying such patients with low levels of detectable viral RNA that straddle the LOD of the assay of choice needs more clarity. It may be challenging to interpret RT-PCR tests for SARS-CoV-2 infection particularly early in the course of infection when these results underlie clinical decision-making to minimize onward transmission.

A relatively lower C T value increases the probability of the presence of infectious virus, but the C T value itself is not enough to determine whether the virus can be cultured from the specimens. Thus a limitation of PCR and other nucleic acid-based diagnostics is that they do not establish infectivity. Studies have shown that viral infectivity, as defined by culturability, is decreased when RT-PCR C T values are >24, with the odds ratio for infectivity being decreased by 32% for every 1-unit increase in C T . 14 Wölfel et al 13 were able to isolate virus during the first week of symptoms particularly from a majority of sputum specimens but could not isolate the virus from specimens taken after day 8. The infectivity of the virus culture does appear to correlate with viral load, and specimens that contain <10 6 copies/mL do not yield an isolate. 13, 15 In addition, Huang et al 15 reported that the mean C T values of culturable oropharyngeal (OP) and NP specimens were similar whereas culturable sputum specimens had relatively lower C T values. Specimens for a successful viral culture contained higher viral RNA counts, but note that viral genome integrity is an important factor for culturability and infectivity assessment. In their study, Huang et al 15 stated that a genome copy number of 5 to 6 log 10 genome copies/mL appears to be the minimal viral load necessary for virus isolation. Detection of SARS-CoV-2 by RT-PCR is still the diagnostic gold standard, and the World Health Organization recommends 2 negative tests at least 24 hours apart as 1 of the key criteria for a clinically recovered patient. 16 In their study, Bullard et al 14 showed a link between in vitro viral growth, C T value, and symptom-to-test. Thus, these authors suggested that a C T value >24 in conjunction with duration of symptoms >8 days could be used together to determine the duration of infectivity in patients.

Nucleic acid amplification kits may sometimes produce false-negative results that do not match clinical features. The false-negative rate is a function of time with virus exposure and subsequent pathogenesis whether infection is presymptomatic, asymptomatic, or paucisymptomatic. In a recent case study, Lv et al 17 reported a successively RT-PCR-negative patient with COVID-19. The authors emphasized the importance of clinical signs and symptoms, other laboratory findings, and chest computed tomography images that should be considered in spite of RT-PCR-negative results. They also reported the dynamic change process of SARS-CoV-2 target genes by RT-PCR testing during the course of infection of a patient with COVID-19 from successive negative results to a successive single positive N gene to 2 positive ORF1a/b and N genes.

If clinical suspicion is high, then infection should not be ruled out on the basis of RT-PCR alone and the clinical and epidemiologic situation should be carefully considered. 18 Because RT-PCR positivity persists beyond the period of infectivity, clinical criteria that include isolation of hospitalized patients for 14 days from symptom onset or 72 hours symptom-free would be the prudent approach. Early discharge followed by home isolation of patients after day 10 of symptoms appears to have little risk of infectivity. 13, 14 Factors such as comorbidities and/or age are certainly part of the consideration in addition to other clinical findings that inform clinical decision-making. Children with SARS-CoV-2 infection have milder clinical symptoms and fewer laboratory and radiologic abnormalities. Nonetheless, understanding the perinatal outcomes of infants born to women infected with SARS-CoV-2 during pregnancy is important for early identification of children with SARS-CoV-2 to provide optimal medical care and control the pandemic. 19 Finally, the CDC advises clinicians to use their judgment to determine if a patient has signs or symptoms compatible with COVID-19 and whether the patient should be tested. Clinicians are encouraged to consider testing for other causes of respiratory illness in addition to testing for SARS-CoV-2 depending on patient age, season, or clinical setting. Further details and updated recommendations may be accessed on the CDC website. 5 Overview www.labmedicine.com Lab Medicine 2020;XX;3-9 LOD of the CDC-LDT assay. The authors highlighted the importance of having multiple viral detection testing platforms available in a public health emergency.

Zhen, Manji, et al 22 stressed the importance of accurate testing results in patient management and, to that end, the importance of false negative results because they inevitably lead to more exposures that can be potentially devastating. Zhen, Smith, et al 23 compared the performance characteristics of 4 PCR methods that yielded comparable results (κ ≥ 0.96); however, they did observe a notable difference when it came to overall LOD, with the GenMark (Carlsbad, CA) having the overall highest LOD of all 4 platforms evaluated and the DiaSorin Simplexa having the lowest LOD (39 ± 23 copies/mL), closely followed by the Hologic Panther Fusion (83 ± 36 copies/mL). Their modified CDC assay showed a final LOD of 779 ± 27 copies/mL. The clinical correlation was also consistent with LOD findings where both the DiaSorin Simplexa and Hologic Panther Fusion assays had 100% positive agreement and detected all specimens deemed positive by the consensus test result, which was 3 of 4 evaluated assays as the gold standard.

As is to be expected, differences exist in LODs across assays, as reported in these studies. More such studies 25, 26 continue to be published with data that will better define the applicability and optimal use of these different assays in laboratory testing to inform clinical decision-making.

Test manufacturers seeking EUA are required to establish performance characteristics such as accuracy, precision, analytical sensitivity, and analytical specificity. The analytical sensitivity of an assay is the ability of an assay to detect very low concentrations of a given analyte in a biological specimen. The LOD is the lowest actual concentration of an analyte that can be consistently detected in typically ≥95% of specimens tested; thus it is an important performance characteristic. Analytical performance at the low concentration limit is critical for infectious disease diagnostics.

As the representative assays presented in Table 1 indicate, there is considerable variability in the manner in which molecular assay LODs are reported in manufacturer IFUs. Per manufacturer IFUs, the claimed LOD is reported as median tissue culture infectious dose (TCID 50 )/mL, copies/ mL, genome copy equivalent/mL, or copies/reaction. The genome equivalent is the amount of DNA necessary to be present in a purified specimen to guarantee that all genes are present.

The calculation for TCID 50 /mL depends on the cell line being used for culture and is based on the degree of cytopathic effect observed in the tissue culture. Viral counts reported by TCID 50 tend to be much lower than RT-quantitative PCR measurements (Bar-On 27 ). The RNA counts can overestimate infectious virions because the presence of viral RNA does not necessarily imply the presence of infectious virions. To assess the concentration of infectious viruses, researchers typically measure TCID 50 , which involves infecting replicate cultures of susceptible cells with dilutions of the virus and noting the dilution at which half the replicate dishes become infected.

Several key variables define what is ultimately the LOD for a particular assay system because individual assay performance characteristics vary. Although LOD can be determined by calculating the point at which a signal can be discriminated from the background, it is more often determined empirically by testing serial dilutions of target analyte such as viral stocks.

In addition to the physicochemical factors that impact assay performance, some assay technologies involve direct specimen testing but most are methods that have a preanalytical specimen extraction step. The direct specimen assay systems include the Diasorin Simplexa Covid-19 Direct Assay on the Liaison MDX and the Atila Biosystems (Mountain View, CA) iAmp COVID-19 Detection Kit, which is run on the Bio-Rad CFX Real-Time System using an isothermal amplification method.

Among these assay systems, there is a variation in the input specimen volume for a given SARS-CoV-2 molecular assay that also affects what is ultimately determined to be the LOD for the assay.

As shown in Table 1 , assay designs vary across the assays and instruments on which they have been validated. The SARS-CoV-2 RNA transcript encodes multiple genes such as the replicase complex ORF1ab, the spike protein, the viral envelope, Overview www.labmedicine.com the membrane, and the nucleocapsid proteins. Although gene targets are selected for their SARS-CoV-2 specificity, the assay designs have included 1 or more gene sequences among the ORf1a/b, nucleocapsid, spike protein, envelope, membrane, and RNA-dependent RNA polymerase genes.

The specificity of gene targets for SARS-CoV-2 is a critical part of assay design. This specificity is necessary for SAR-CoV-2 detection and discrimination among other related SARS viruses. Coronaviruses have a significantly lower mutation rate than those observed in other RNA viruses, such as the influenza virus, because of the presence of intrinsic proofreading activity during viral replication. [28] [29] [30] [31] [32] [33] It is possible that successful SARS-CoV-2 replication cycles may accumulate mutations that could ultimately contribute to differences in clinical outcomes between patients with differing viral populations. Gussow et al 32 and Artesi et al 34 identified a C-to-U transition at position 26,340 of the SARS-CoV-2 genome that spans the envelope gene. Moreover, they stressed the importance of targeting 2 genomic regions to minimize the chance for false negative results. It is thus prudent to monitor circulating SARS-CoV-2 variants that may adversely impact the performance of RT-PCR diagnostic assays. 35 

Currently, the assays with EUA are all qualitative, providing a binary positive or negative answer (or a third answer as presumptive positive or indeterminate in a few assay systems). Most RT-PCR assay systems are set up for a C T value <40 to be clinically reported as PCR-positive. However, this situation does not always occur because some assay systems continue amplification cycles beyond 40. It is likely that a patient test result that may be positive has a late C T (eg, >40 C T ) and may remain undetected on a different assay platform. Nevertheless, a PCR-positive result indicates the presence of viral RNA and may not necessarily indicate the presence of viable virus. Thus, it is difficult to correlate the C T values from one assay system to another in an accurate manner. 36, 37 It is assumed during PCR that each template molecule is duplicated once per cycle. However, for templates expressed at very low levels variance exists in the results from PCR reactions with scattered C T values between replicate PCR reactions. 38 As can be envisaged for low copy number templates, comparison across different assay designs, instruments, target genes, and PCR physicochemical characteristics is difficult.

The SARS-CoV-2 molecular tests discussed are performed after sampling using NP swabs or other upper respiratory tract specimens, including OP swabs or saliva. Aside from the diversity of assay designs, target genes, molecular methodologies, and platforms used to detect SARS-CoV-2, specific specimen types, which include NP and OP swabs, BAL, and sputum impact the LOD of an assay. In addition, alternate specimen types such as saliva and cotton swabs have also been validated for specific assays and intended uses, as has the use of saline as an alternative transport medium vs conventional viral transport medium. Limited allocations of swabs (and reagents) have compelled manufacturers (and clinical laboratories) to validate alternate swabs and transport to be used for patient specimen collection and testing. 39 The timeline of testing for PCR positivity may vary with the type of specimen. However, BAL fluid specimens seem to show the highest positive rates, followed by sputum, nasal swabs, fibrobronchoscope brush biopsy, pharyngeal swabs, feces, and blood. 40 False-negative results may occur because of inappropriate timing of specimen collection in relation to illness onset and deficiency in sampling technique, especially of NP swabs. Research has shown that PCR positivity declines more slowly in sputum and may still be positive after NP swabs are negative. The persistence of PCR in sputum and stool has been found to be similar as assessed. 13

There is a dire need for the development of international standards for a SARS-CoV-2 RNA copy number. Development of such standards would permit comparison of the myriad assay systems such that the assay LOD can be reported in a standardized manner. This would be very useful for clinical practitioners and laboratory and other healthcare professionals.

To support its evaluation of diagnostic tests for COVID-19, the FDA has announced that it will provide a SARS-CoV-2 reference panel to ensure the quality of the tests, validation of new assays, test calibration, and monitoring of assay performance. The FDA panel will be available to commercial and laboratory developers who are interacting with the FDA through the pre-EUA process.

The Coronavirus Standards Working Group, bringing together commercial, academic, and public-sector interests, has been formed to develop tools that ensure the reliability and accuracy of this vital COVID-19 testing. 41 As stated on its website, this working group will develop an annotated inventory of resources, identify gaps, and conduct international collaborative experiments to evaluate and establish comparability of the diverse control materials and test methods available. A commendable objective is to pool knowledge to collaboratively produce the most-needed reference materials, reference samples, and reference measurement methods to underpin an enduring and reliable COVID testing enterprise.

Bustin and Nolan 42 stated in 2020 that current RT-PCR testing for SARS-CoV-2 has several shortcomings because of the shortage in the availability of suitable assays, reagents, equipment, and testing capacity. Interestingly, they recommended centralized RNA extraction procedures in containment level 3 facilities with dissemination of analytical testing of noninfectious RNA to nationwide laboratories. They suggested that these procedures be considered by public health organizations such as the CDC and Public Health England in the United Kingdom for protocols for emergency testing systems that can be rapidly adopted in emergency situations.

Worldwide multifarious assay systems are currently in use to test for SARS-CoV-2. To effectively implement clinical countermeasures, it is imperative to recognize the clinical decision-making that may be influenced by variation in the performance characteristics of the diverse assay systems in use for SARS-CoV-2 diagnostic testing. The assay results, particularly those that fall near the LOD, may not be commutable from one platform to another. As discussed herein, efforts to standardize across the different assays, platforms, and technologies would enhance clinical diagnostic testing to the ultimate benefit of patients. It is anticipated that cross-platform comparison studies will continue to add to our knowledge base. Simultaneous ongoing efforts to learn more about SARS-CoV-2 pathogenesis would lead to improved diagnostic testing such that successful preventive, diagnostic, and prognostic countermeasures may be implemented for controlling the ongoing pandemic. LM

19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas#individual-molecular. Individual EUAs for molecular diagnostic tests for SARS-CoV-2. U.S Food & Drug Administration. Last updated

Clinical evaluation and utilization of multiple molecular in vitro diagnostic assays for the detection of SARS-CoV-2

Comparison of commercially available and laboratory developed assays for in vitro detection of SARS-CoV-2 in clinical laboratories

Asymptomatic SARS-CoV-2 infections: What do we need to know? Infect Control Hosp Epidemiol

Overview of testing for SARS-CoV-2 (COVID-19)

SARS-CoV-2 viral load in upper respiratory specimens of infected patients

Temporal dynamics in viral shedding and transmissibility of COVID-19

Prolonged persistence of SARS-CoV-2 RNA in body fluids

Shedding of infectious virus in hospitalized patients with coronavirus disease-2019 (COVID-19): duration and key determinants

Positive RT-PCR test results in patients recovered from COVID-19

The presence of SARS-CoV-2 RNA in the feces of COVID-19 patients

Persistent viral shedding of SARS-CoV-2 in faeces-a rapid review

Virological assessment of hospitalized patients with COVID-2019

Predicting infectious SARS-CoV-2 from diagnostic samples

Culture-based virus isolation to evaluate potential infectivity of clinical specimens tested for COVID-19

Public health surveillance for COVID-19: interim guidance. World Health Organization

Dynamic change process of target genes by RT-PCR testing of SARS-Cov-2 during the course of a coronavirus disease 2019 patient

Variation in false negative rate of RT-PCR based SARS-CoV-2 tests by time since exposure

Coronavirus infections in children including COVID-19. An overview of the epidemiology, clinical features, diagnosis, treatment and prevention options in children

Assay techniques and test development for COVID-19 diagnosis

High-throughput transcriptionmediated amplification on the Hologic Panther is a highly sensitive method of detection for SARS-CoV-2

Comparison of four molecular in vitro diagnostic assays for the detection of SARS-CoV-2 in nasopharyngeal specimens

Clinical evaluation of three sample-toanswer platforms for the detection of SARS-CoV-2

Comparison of two high-throughput reverse transcription-polymerase chain reaction systems for the detection of severe acute respiratory syndrome coronavirus 2

Comparison of Abbott ID Now, DiaSorin Simplexa, and CDC FDA EUA methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from individuals diagnosed with COVID-19

Comparison of Abbott ID Now and Abbott m2000 methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from symptomatic patients

SARS-CoV-2 (COVID-19) by the numbers

Coronaviruses: an RNA proofreading machine regulates replication fidelity and diversity

Coronaviruses: an overview of their replication and pathogenesis

Viral mutation rates

Discovery of an RNA virus exoribonuclease that is critically involved in coronavirus RNA synthesis

Genomic determinants of pathogenicity in SARS-CoV-2 and other human coronaviruses

SARS-CoV-2 is mutating slowly, and that's a good thing

A recurrent mutation at position 126340 of SARS-CoV-2 is associated with failure of the E-gene qRT-PCR utilized in a commercial dual-target diagnostic assay

D614G mutation of SARS-CoV-2 spike protein enhances viral infectivity

Interpreting diagnostic tests for SARS-CoV-2

Comparative performance of SARS-CoV-2 detection assays using seven different primer/probe sets and one assay kit

Data analysis and interpretation

Evaluation of saline, phosphate buffered saline and minimum essential medium as potential alternatives to viral transport media for SARS-CoV-2 testing

Detection of SARS-CoV-2 in different types of clinical specimens

Coronavirus Standards Working Group: a COVID-19 diagnostic standards development partnership. The Joint Initiative for Metrology in Biology

RT-qPCR testing of SARS-CoV-2: a primer