key: cord-0026766-zapxked1
authors: Simner, Patricia J; Rauch, Carol A; Martin, Isabella W; Sullivan, Kaede V; Rhoads, Daniel; Rolf, Robin; She, Rosemary; Souers, Rhona J; Wojewoda, Christina; Humphries, Romney M
title: Raising the Bar: Improving Antimicrobial Resistance Detection by Clinical Laboratories by Ensuring Use of Current Breakpoints
date: 2022-02-07
journal: Open Forum Infect Dis
DOI: 10.1093/ofid/ofac007
sha: 0ca554d7247aee251c3cc0552f22203adfc3e433
doc_id: 26766
cord_uid: zapxked1

BACKGROUND: Antimicrobial resistance (AMR) is a pressing global challenge detected by antimicrobial susceptibility testing (AST) performed by clinical laboratories. AST results are interpreted using clinical breakpoints, which are updated to enable accurate detection of new and emerging AMR. Laboratories that do not apply up-to-date breakpoints impede global efforts to address the AMR crisis, but the extent of this practice is poorly understood. METHODS: A total of 1490 clinical laboratories participating in a College of American Pathologists proficiency testing survey for bacterial cultures were queried to determine use of obsolete breakpoints. RESULTS: Between 37.9% and 70.5% of US laboratories reported using obsolete breakpoints for the antimicrobials that were queried. In contrast, only 17.7%–43.7% of international laboratories reported using obsolete breakpoints (P < .001 for all comparisons). Use of current breakpoints varied by AST system, with more laboratories reporting use of current breakpoints in the US if the system had achieved US Food and Drug Administration clearance with current breakpoints. Among laboratories that indicated use of obsolete breakpoints, 55.9% had no plans to update to current standards. The most common reason cited was manufacturer-related issues (51.3%) and lack of internal resources to perform analytical validation studies to make the update (23.4%). Thirteen percent of laboratories indicated they were unaware of breakpoint changes or the need to update breakpoints. CONCLUSIONS: These data demonstrate a significant gap in the ability to detect AMR in the US, and to a lesser extent internationally. Improved application of current breakpoints by clinical laboratories will require combined action from regulatory agencies, laboratory accreditation groups, and device manufacturers.

Antimicrobial resistance (AMR) is a pressing global problem, associated with >700 000 deaths annually, a number projected to increase to 10 million by 2050 if left unchecked [1] . Key to confronting AMR is the detection of antibiotic resistance by clinical laboratories-a task that requires accurate interpretation of antimicrobial susceptibility testing (AST) results [2, 3] . Over the past decade, AST interpretive criteria (also known as breakpoints) have evolved in response to emerging AMR mechanisms, improved understanding of pharmacokinetics and pharmacodynamics, and new clinical data indicating that previous breakpoints were incorrect (Table 1) [5, 6] . A breakpoint enables interpretation and reporting of an AST result as "susceptible," "intermediate," "susceptible-dose-dependent, " or "resistant," which guides clinical therapeutic decision-making. To date, breakpoint updates have involved lowering susceptibility breakpoints; for example, the susceptible breakpoint for ceftriaxone against Enterobacterales was a minimum inhibitory concentration (MIC) of ≤8 µg/mL prior to 2010, whereas the current susceptible breakpoint is ≤1 µg/mL. This change was made as new clinical data demonstrated poor outcomes for patients treated with ceftriaxone for isolates with MICs of 2, 4, or 8 µg/mL [7, 8] . Breakpoint changes can benefit patients and public health only if adopted in a timely manner by diagnostic companies that develop and market AST products and/or clinical laboratories that perform AST. Currently, the regulatory framework in the United States (US) does not compel laboratories or diagnostic companies to keep up to date with evolving AST breakpoints [9] . In the absence of regulatory stimulus for laboratories and manufacturers, the need to update breakpoints goes mostly unrecognized and is underprioritized due to financial burdens and lack of other resources. As a result, clinical laboratories may interpret AST results using breakpoints that have been obsolete for a decade or more [9] . This results in serious patient safety concerns and hampers the ability to track and contain the worldwide threat of AMR.

The College of American Pathologists (CAP) accredits laboratories globally, with the goals of promoting patient safety and advancing excellence in the practice of laboratory medicine. A key service provided by the CAP is the worldwide administration of proficiency testing programs, to ensure that laboratories achieve accurate testing results. Over the past several years, CAP Microbiology Committee members, who serve as subject-matter experts, have become increasingly concerned that clinical laboratories may be applying obsolete breakpoints to AST results based on proficiency testing results, which reflect what would be reported to patient-facing clinicians in similar circumstances. Data from proficiency test surveys have shown that use of obsolete breakpoints results in misclassification of bacterial isolates as susceptible when they were in fact resistant to tested antimicrobial agents. The present study quantified the extent of the use of obsolete breakpoints among US and international clinical laboratories that participate in a CAP proficiency testing program for bacterial testing. Table  1 ) and distributed to 2296 laboratories in June 2019 (1873 US laboratories and 423 international laboratories). International laboratories included those from 76 countries (Supplementary  Table 2 ).

In the questionnaire, current and obsolete breakpoints were listed for 7 combinations of organism/antimicrobial agents ( Table 1 ) that had been updated since 2010. All "current" breakpoints were recognized by both the Clinical and Laboratory Standards Institute and the US Food and Drug Administration (FDA) at the time of the questionnaire ( Table 1 ). Laboratories that indicated use of obsolete breakpoints or were unsure which breakpoints they used were asked to enter a free text comment describing the primary reason for this status. The free text comments were combined into categories and stratified by location of the laboratory (US or international). There was one adjustment applied to the breakpoint responses: Other/ unsure responses (n = 80-121, dependent on breakpoint) that were submitted without additional free text responses clarifying their practice were treated as unsure and excluded, as were "not tested" responses. Participants were also asked to specify the primary test system used for AST and divided into commercially available automated or manual methods. Automated systems were defined as AST systems that consist of automated inoculation of MIC panels followed by computer-assisted incubation with reading, interpretation, and reporting functions that do not require manual intervention. These included the commercial AST systems Phoenix (Becton Dickinson and Company, Sparks, Maryland), MicroScan (Beckman Coulter, Sacramento, California), and Vitek 2 (bioMérieux, Durham, North Carolina). Manual methods require manual intervention for setup, reading, and interpretation. These included disk diffusion, gradient diffusion, broth dilution, and agar dilution methods. Two testing factors were defined for the statistical analysis. A 2-level location factor was defined as US or international, and a 3-level AST system factor was based on the major commercial AST systems: BD Phoenix, Beckman Coulter MicroScan, and bioMérieux Vitek 2. A multivariate logistic regression model was used to test for breakpoint usage differences between location type (US or international) for all 7 organism/antimicrobial combinations. A second multivariate model to evaluate breakpoint usage differences across all organism/antimicrobial combinations was fit with both the location and AST system factors in addition to the interaction term. Both models used a Bonferroni correction for the multiple tests, and adjusted P values are reported. All analyses were performed with SAS version 9.4 software (SAS Institute, Cary, North Carolina). A significance level of .05 was used for testing.

This study was a survey of laboratory practices and did not include factors necessitating patient consent.

Overall, 2296 laboratories participated in the D-B 2019 bacteriology proficiency testing challenge, and 1490 laboratories (64.9%; range, 966-1490 dependent on breakpoint queried) provided responses to the supplemental questionnaire. Of these respondents, there were 1258 from the US and 232 from international locations with a response rate of 67.2% and 54.8%, respectively. Laboratories reported the test methodology used for each antimicrobial ( Table 2 ). Regardless of location of the laboratory (US or international), the majority indicated use of automated AST methods, with the remaining applying manual methods (eg, disk diffusion, gradient diffusion, and manual MIC tests).

Between 29.5% and 62.1% of US laboratories reported using current breakpoints for antimicrobials that were queried (Table 3 ). In contrast, 56.3%-82.3% of international laboratories reported using current breakpoints. A significantly higher proportion of international laboratories reported applying current breakpoints to AST results for all organism(s)/ antimicrobial agent combinations compared to US laboratories (P < .001).

More laboratories reported application of current Enterobacterales breakpoints for ceftazidime (US, 59.3%; international, 81.6%), ceftriaxone (US, 61.7%; international, 82.3%), and meropenem (US, 62.1%; international, 79.7%), all of which were updated in 2010, than for ciprofloxacin (US, 29.5%; international, 59.2%) and levofloxacin (US, 30.0%; international, 56.3%), which were updated in 2019 (Table 3) . Up to 12.5% of laboratories for each breakpoint queried indicated they were unsure of the breakpoint applied by their laboratory (not shown). Use of current breakpoints was further analyzed according to automated AST device (Table 4 ). Phoenix users, both in the US and internationally, were more likely to apply current breakpoints than laboratories using MicroScan or Vitek 2, across all antimicrobials evaluated (P < .01 for all combinations). In addition, more US laboratories using MicroScan applied current breakpoints for ceftazidime (P = .04), ciprofloxacin (P < .001), and levofloxacin (P < .001) than those using Vitek 2.

Across all breakpoints evaluated, a higher proportion of international laboratories reported applying current breakpoints compared to US laboratories for each automated AST device in use. These differences were statistically significant for Vitek 2 users comparing US vs international laboratories for ceftazidime (P < .001), ceftriaxone (P < .001), ciprofloxacin (P < .01), levofloxacin (P < .005), piperacillintazobactam (P < .001), and imipenem (P < .01) breakpoints. Similarly, among laboratories using MicroScan, international laboratories more often applied current breakpoints for ciprofloxacin (P < .001) and levofloxacin (P = .002) compared to US laboratories. Among laboratories using the Phoenix system, more international laboratories applied current ciprofloxacin breakpoints than US laboratories using this system (P = .04).

Of the participants who had not updated to current breakpoints or were unsure of the breakpoints they applied, 781 Ceftazidime: Phoenix -MicroScan, P = .003; MicroScan -Vitek 2, P = .04.

Ceftriaxone: Phoenix -MicroScan, P < .001; Phoenix -Vitek 2, P < .001.

Ciprofloxacin: Phoenix -MicroScan, P = .003; MicroScan -Vitek 2, P < .001.

Levofloxacin: Phoenix -MicroScan, P = .004; MicroScan -Vitek 2, P < .001.

Meropenem: Phoenix -MicroScan, P < .001; Phoenix -Vitek 2, P = .002.

Piperacillin-tazobactam: Phoenix -MicroScan, P < .001; Phoenix -Vitek 2, P < .001.

Imipenem: Phoenix -MicroScan, P < .001; Phoenix -Vitek 2, P < .001. b Significant system differences between US and international laboratories:

Ceftazidime: Vitek 2, P < .001.

Ceftriaxone: Vitek 2, P < .001.

Ciprofloxacin: Phoenix, P = .04; MicroScan, P < .001; Vitek 2, P < .001.

Levofloxacin: MicroScan, P = .002; Vitek 2, P = .005.

Piperacillin-tazobactam: Vitek 2, P < .001.

Imipenem: Vitek 2, P < .001.

laboratories (52.4% of those responding to the supplemental questionnaire) entered 974 comments explaining the rationale. Some laboratories provided multiple comments with different explanations based on the organism/antimicrobial agent combination queried. Fifty-six comments associated with US laboratories were excluded from the analysis due to unclear interpretation or relevance of the comment. Overall, 44.1% of respondents indicated that efforts to implement current breakpoints were underway, whereas 55.9% indicated that they did not have current plans to update (Table 5 ). One strategy used by 27.4% of US laboratories and 78.8% of international laboratories was to not perform testing for certain drugs, but to send the test to a reference laboratory. Other laboratories not performing testing using their primary AST method explained that they used an alternate test method, which could include AMR mechanism testing. The most common reason cited by laboratory respondents for continued use of obsolete breakpoints was manufacturer-related issues (51.3%), followed by lack of internal resources to perform analytical validation studies (23.4%). A small fraction of laboratories (13.3%) indicated that they were unaware of breakpoint changes or the need to update breakpoints. Four US laboratories indicated that they did not intend to update their breakpoints. Due to insufficient number of responses across test systems, we did not attempt to correlate these responses to trends between AST system manufacturers.

The challenge of AMR threatens one of the greatest successes of modern medicine [10, 11] . In 2019, the US Centers for Disease Control and Prevention declared that the US had entered the postantimicrobial era [11] . The data presented herein highlight a serious patient safety and public health problem-the widespread struggle by clinical laboratories to stay up to date with current breakpoints. Use of obsolete breakpoints leads to interpretation of organisms as susceptible to an antimicrobial when they are actually resistant [12] , guiding the treating clinician to use ineffective antimicrobials and putting the patient's safety at risk [6] . More broadly, use of obsolete breakpoints limits the global public health response to AMR, as pathogens of serious or urgent concern can go undetected, spreading to additional patients and across healthcare systems and communities [13, 14] . Improved awareness, oversight, and regulation of AST is needed [4, 15] . The reasons for why laboratories struggle with this task are complex, but certainly relate to the fact that diagnostic manufacturers are not required to update breakpoints on FDA-cleared systems. In addition, many laboratories commonly assume that use of an FDA-cleared AST system is both necessary and sufficient to assure quality results, resulting in the high fraction providing comments related to "manufacturer-related issues. " This study demonstrated that laboratories outside the US were more likely to apply contemporary breakpoints than laboratories in the US. One contributing factor may relate to US-specific regulations imposed on diagnostic test manufacturers and clinical laboratories alike. AST devices marketed in the US must apply the most up-to-date FDA-recognized breakpoints at the time of initial clearance, but there is currently no FDA regulatory requirement that manufacturers update their systems as breakpoints change. As such, devices that were cleared >10 years ago may continue to apply obsolete, pre-2010 breakpoints [15] . Compounding this dilemma, the US Government Accountability Office found in 2020 that FDA had limited knowledge of what breakpoints were applied by FDA-cleared AST devices (https://www.gao.gov/products/ GAO-20-341). Updating breakpoints on commercially available AST devices requires substantial financial commitment for manufacturers, as new data need to be generated to obtain clearance for an updated breakpoint on an existing AST device. Companies have indicated that updating breakpoints on their devices also comes at significant opportunity cost, slowing the development of more rapid and accurate tests (https://wayback. archive-it.org/7993/20201222130145/https://www.fda.gov/ medical-devices/workshops-conferences-medical-devices/ public-workshop-antimicrobial-susceptibility-and-resistanceaddressing-challenges-diagnostic-devices). Despite these challenges, some AST companies have voluntarily prioritized updating breakpoints on their automated AST devices. We showed that in the US, use of current breakpoints was more common in Phoenix users than in laboratories that use Vitek 2 or MicroScan. At the time of the survey, the Phoenix system had obtained FDA clearance for the majority of the breakpoints queried in the survey, whereas clearance on the other 2 devices was less consistent (https://www.fda.gov/medical-devices/ device-approvals-denials-and-clearances/510k-clearances). Availability of tests that are FDA-cleared with current breakpoints would clearly improve the status of AMR detection in the US. The FDA has indicated that market incentives (ie, customer demand) should be sufficient to drive manufacturers to ensure devices are updated (https://www.gao.gov/products/ GAO-20-341). However, this concept is difficult to apply as many laboratories are unaware of the problem [16] ; no competitor product(s) are available that apply updated breakpoints to all antimicrobials tested, and changing AST devices is a daunting task for clinical laboratories given the multiple species and antimicrobials tested in routine practice. The FDA has developed a streamlined regulatory pathway to facilitate more timely updating of US AST devices to include new antimicrobial agents, but older and more commonly used antimicrobials (including those evaluated in this report) are not eligible for this pathway. The rationale for this difference is unclear. Outside the US, manufacturers may update breakpoints on AST devices without seeking additional formal approval from regulatory bodies. In Europe, the European Medicines Agency granted breakpoint setting authority to the European Committee on Antimicrobial Susceptibility Testing, resulting in a single, unified set of breakpoints, which further streamlines the processes. Several manufacturers market the same test system in the US with obsolete breakpoints that is available outside the US with up-to-date breakpoints, due to these regulatory challenges. Until now, no regulatory bodies that accredit Clinical Laboratory Improvement Amendments (CLIA)-licensed laboratories have required up-to-date breakpoints, but the CAP newly requires CAP-accredited laboratories to maintain up-todate breakpoints, effective 2024. By this new requirement, laboratories may either adopt a methodology that is up to date or validate their current systems for updated breakpoints. While the CAP is reluctant to force laboratories to use FDA-cleared devices in an off-label manner, ensuring patient safety through the use of current breakpoints is of the utmost importance. CLIA regulations allow laboratories to make such modifications to AST devices, provided the laboratory performs analytical studies to validate the modification prior to implementation. Many laboratories that use obsolete breakpoints cited lack of technical expertise and laboratory resources to pursue the validation studies necessary to update their breakpoints, but many as 62% of US laboratories indicated that they used current breakpoints, indicating that many have taken this step. Nonetheless, institutional leadership and commitment of resources is needed to ensure that all laboratories prioritize and implement breakpoint updates in a timely manner [17, 18] . This report has several limitations. First, we relied on laboratory self-reporting of breakpoint use. Data from this questionnaire show that up to 10% of laboratories were unaware of which breakpoints are applied by their AST devices. Furthermore, many laboratories indicated that they (incorrectly) assumed breakpoints were automatically updated on FDA-cleared devices. Not all laboratories that participated in the proficiency test challenge provided responses to the questionnaire, so some self-selection bias is likely present in the data (ie, laboratories that applied obsolete breakpoints may have been less likely to participate in the questionnaire). Second, this questionnaire was limited to laboratories accredited by CAP, which includes 44% of clinical laboratories in the US (https://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA/Downloads/statupda.pdf). CAPaccredited laboratories are held to a higher standard than what is required by CLIA as a minimum standard for clinical laboratories. Only a subset of the most impactful breakpoint changes was included in the questionnaire to make it manageable for participants. As such, the true extent of obsolete breakpoint use may be underrepresented in this report. Representation of laboratories outside the US was limited, although the sampling was sufficient to calculate differences between US and international laboratory practices. Finally, since this report, updates have been made to some of the systems discussed herein. Strengths of this survey, however, include broad representation of US laboratories.

In summary, we highlight that AMR detection in the US is significantly hampered by the continued application of obsolete breakpoint interpretations to AST results by clinical laboratories. Advocacy for additional resources and support from all levels within hospital systems, regional and state public health laboratories, and the federal government is needed to enable commercial manufacturers and clinical laboratories to consistently ensure that AST results are up to date with current breakpoints. If the local laboratory applies obsolete breakpoints to AST results, patient-facing clinicians should be made aware by the laboratory that such breakpoints may lead to incorrect test results for a given patient, and infection control and prevention staff must also be made aware that laboratory identification of antimicrobial-resistant bacteria may be suboptimal. Communication with the laboratory and the AST device manufacturer is critical to understand what standards are applied to AST results for interpretation. The US National Action Plan for Combating Antibiotic-Resistant Bacteria [16, 19] calls for federal agencies to strengthen surveillance and advance the development of diagnostic tests (including AST devices) and new antimicrobials in order to slow the emergence of antimicrobialresistant organisms. While the results of this survey are concerning and disappointing, it is possible to improve AMR testing through the combined efforts of industry, government, standards development organizations, and clinical laboratories.

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Financial support. This work was supported by the College of American

Review on Antimicrobial Resistance. Tackling drug-resistant infections globally: final report and recommendations

Antimicrobial susceptibility testing: a review of general principles and contemporary practices

Emerging resistance, new antimicrobial agents ... but no tests! the challenge of antimicrobial susceptibility testing in the current US regulatory landscape

Understanding and addressing CLSI breakpoint revisions: a primer for clinical laboratories

The Clinical and Laboratory Standards Institute subcommittee on antimicrobial susceptibility testing: background, organization, functions, and processes

Clinical outcomes of Enterobacteriaceae infections stratified by carbapenem MICs

Rationale for the 2010 revised susceptibility breakpoints for cephalosporins, aztreonam, and carbapenems for Enterobacteriaceae

Background and rationale for revised Clinical and Laboratory Standards Institute interpretive criteria (breakpoints) for Enterobacteriaceae and Pseudomonas aeruginosa: I. Cephalosporins and aztreonam

Carbapenem-resistant enterobacteriaceae detection practices in California: what are we missing?

Antibiotic consumption and stewardship at a hospital outside of an early coronavirus disease 2019 epicenter

COVID-19, superinfections and antimicrobial development: what can we expect?

Breakpoint beware: reliance on historical breakpoints for Enterobacteriaceae leads to discrepancies in interpretation of susceptibility testing for carbapenems and cephalosporins and gaps in detection of carbapenem-resistant organisms

Antibiotic resistance threats in the United States

Impact of delays between Clinical and Laboratory Standards Institute and Food and Drug Administration revisions of interpretive criteria for carbapenem-resistant Enterobacteriaceae

Impact of 21st Century Cures Act on breakpoints and commercial antimicrobial susceptibility test systems: progress and pitfalls

National action plan for combating antibiotic-resistant bacteria

Verification is an integral part of antimicrobial susceptibility test quality assurance

College of American Pathologists (CAP) microbiology committee perspective: the need for verification studies

National action plan for combating antibiotic-resistant bacteria

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.