key: cord-1044997-394fwzv3 authors: Willis, T. A.; Wood, S.; Brehaut, J.; Colquhoun, H.; Brown, B.; Lorencatto, F.; Foy, R. title: Opportunities to improve the impact of two national clinical audit programmes: a theory-guided analysis date: 2022-03-21 journal: Implement Sci Commun DOI: 10.1186/s43058-022-00275-5 sha: bcc20d049551f85c1d6667e340cc4ee76dbbd50d doc_id: 1044997 cord_uid: 394fwzv3 BACKGROUND: Audit and feedback is widely used in healthcare improvement, with evidence of modest yet potentially important effects upon professional practice. There are approximately 60 national clinical audit programmes in the UK. These programmes often develop and adapt new ways of delivering feedback to optimise impacts on clinical practice. Two such programmes, the National Diabetes Audit (NDA) and the Trauma Audit Research Network (TARN), recently introduced changes to their delivery of feedback. We assessed the extent to which the design of these audit programmes and their recent changes were consistent with best practice according to the Clinical Performance Feedback Intervention Theory (CP-FIT). This comprehensive framework specifies how variables related to the feedback itself, the recipient, and the context operate via explanatory mechanisms to influence feedback success. METHODS: We interviewed 19 individuals with interests in audit and feedback, including researchers, audit managers, healthcare staff, and patient and public representatives. This range of expert perspectives enabled a detailed exploration of feedback from the audit programmes. We structured interviews around the CP-FIT feedback cycle and its component processes (e.g. Data collection and analysis, Interaction). Our rapid analytic approach explored the extent to which both audits applied features consistent with CP-FIT. RESULTS: Changes introduced by the audit programmes were consistent with CP-FIT. Specifically, the NDA’s increased frequency of feedback augmented existing strengths, such as automated processes (CP-FIT component: Data collection and analysis) and being a credible source of feedback (Acceptance). TARN’s new analytic tool allowed greater interactivity, enabling recipients to interrogate their data (Verification; Acceptance). We also identified scope for improvement in feedback cycles, such as targeting of feedback recipients (Interaction) and feedback complexity (Perception) for the NDA and specifying recommendations (Intention) and demonstrating impact (Clinical performance improvement) for TARN. CONCLUSIONS: The changes made by the two audit programmes appear consistent with suggested best practice, making clinical improvement more likely. However, observed weaknesses in the feedback cycle may limit the benefits of these changes. Applying CP-FIT via a rapid analysis approach helps identify strengths and remediable weaknesses in the design of audit programmes that can be shared with them in a timely manner. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s43058-022-00275-5. Contributions to the literature • Audit programmes often make incremental changes to their methods to increase their impacts on clinical practice. • This study demonstrates the practical application of Clinical Performance Feedback Intervention Theory (CP-FIT), using rapid analysis of expert interviews, to two national audit programmes which had recently introduced changes to their feedback methods. • Such changes can augment feedback methods in ways that are consistent with best practices according to CP-FIT. However, their impacts may be limited by a range of remediable weaknesses in the feedback cycle. Audit and feedback is widely used to improve the quality of healthcare internationally, including approximately 60 national clinical audit programmes in the United Kingdom (UK) [1] . It involves reviewing performance against explicit standards (audit) and presenting data to guide improvement (feedback). It is an intervention that can be delivered at scale and at a comparatively low cost, and a Cochrane review of 140 randomised trials showed it produced a median 4.3% absolute improvement in professional practice [2] . Knowledge of the optimal components of audit and feedback can help to make its effectiveness more consistent, with potentially substantial impacts upon clinical practice (in a quarter of trials, the improvement exceeded 16%). Evidence-based methods to enhance audit and feedback are known, e.g. incorporating goals and action plans [2] . However, evolving theory in this field may also suggest ways of improving effectiveness. The majority of audit and feedback interventions in research studies do not explicitly draw upon theory [2, 3] . The extent to which the organisation and delivery of 'real-life' national audit programmes reflect best practice from theory is less known. The Clinical Performance Feedback Intervention Theory (CP-FIT) offers the most comprehensive theory to date on the conditions for optimal audit and feedback [4] . It builds upon 30 existing behaviour change theories and frameworks, providing an integrated theory of feedback specifically relevant to healthcare. It proposes that effective audit and feedback is a cyclical process consisting of goal setting and audit, feedback message production, perception and Acceptance of feedback message, recipient Intention to respond, action (at the individual and organisational levels), and, ultimately, care quality improvement (Fig. 1) . Progress through the cycle will be weakened, or halted entirely, if any individual component fails. We applied CP-FIT to two UK national clinical audit programmes (summarised in Table 1) , which recently introduced changes to their feedback delivery. The National Diabetes Audit (NDA) [5] increased the frequency of data release to primary care recipients from annual to quarterly intervals, consistent with a suggestion to provide feedback as soon as possible following data collection [6] . The Trauma Audit Research Network (TARN) [7] introduced a new system ('TARN Analytics') that offers users greater interactivity to explore data, consistent with a suggestion to provide short, actionable messages followed by optional detail [6] . We drew upon CP-FIT to apply a theoretical lens to evaluate two national audit programmes and hence identify their strengths and opportunities for improvement. We had originally designed a larger-scale evaluation of the two audit programmes, including observational research and interviews with service commissioners, clinicians, and managers. However, the coronavirus pandemic resulted in the suspension of all non-essential UK healthcare research. We therefore completed a pragmatic, non-intrusive evaluation, within the time limits of a funded research programme. We drew upon semi-structured, expert interviews, using CP-FIT as a framework. We used a rapid approach to analysis and interpretation. Rapid techniques within qualitative analysis are considered appropriate to provide valid findings and recommendations within a short timeframe [8] [9] [10] [11] . Our report follows the COREQ recommendations for qualitative studies [12] (Additional file 1). We aimed for a sample of up to 20 individuals from our existing network of co-investigators, collaborators, and advisers, all of whom were directly connected to our National Institute for Health Research-funded study to optimise feedback from national clinical audits. This included audit and feedback researchers, national clinical audit leads and commissioners, clinicians targeted by national audits, and patient and public representatives. We approached individuals informed by their experience and knowledge in clinical audit research and practice, seeking representation across the various roles and backgrounds available. Intervention Theory (CP-FIT) (Fig. 1 [4 CP-FIT offers a comprehensive explanation of how clinical feedback is believed to work and the factors that influence its effectiveness. This includes three categories of variables, relating to the feedback itself (e.g. how the feedback is displayed and delivered), the recipient (e.g. characteristics of the targeted health professionals), and Table 1 Description of the two national clinical audits National Diabetes Audit (NDA) [5] The NDA Programme comprises four modules: the National Diabetes Core Audit, the National Pregnancy in Diabetes Audit, the National Diabetes Footcare Audit, and the National Inpatient Diabetes Audit. The NDA helps improve the quality of diabetes care by enabling participating NHS services and organisations to assess local practice against the NICE guidelines (e.g. the proportion of eligible patients with diabetes that achieve target levels of blood pressure, cholesterol, and blood sugar), compare their care and outcomes with similar services and organisations, identify gaps or shortfalls that are priorities for improvement, identify and share best practice, and provide comprehensive national pictures of diabetes care and outcomes in England and Wales. Audit reports provide national-and local-level information on, for example, prevalence, care process completion, and treatment target achievement. Our study focused on the quarterly release of data included in the Core Audit. [7] TARN is the National Clinical Audit for traumatic injury and is the largest European Trauma Registry, holding data on over 800,000 injured patients including over 50,000 injured children. TARN aims to measure and monitor the processes and outcomes of care (e.g. the proportion of patients with head injury receiving a CT scan within 60 min) to demonstrate the impact of trauma networks, providing local, regional, and national information on trauma patient outcomes, and thereby help clinicians and managers to improve trauma services. Individual patient data are inputted manually at the trauma unit to an online data collection and validation system, aiming to be available within 25 days of patient discharge or death. Our study focused on the online, 'TARN Analytics' tool: a reporting tool designed to offer users a more dynamic method of viewing and manipulating their data, e.g. by supporting the creation and sharing of data visualisations. the context (e.g. the existence of and interaction with other improvement interventions). These variables are proposed to operate via a set of seven explanatory mechanisms to influence the feedback cycle. For instance, 'credibility' refers to the feedback's perceived trustworthiness and reliability; users are more likely to engage with credible feedback. CP-FIT also presents a set of evidence-based, 'high-confidence' hypotheses on how to enhance the effectiveness of feedback. For example, the 'action planning' hypothesis reads, 'feedback interventions are more effective when they provide solutions to suboptimal performance (or support recipients to do so)'; the 'ownership' hypothesis proposes, 'feedback interventions are more effective when recipients feel they "own" it, rather than it has been imposed on them. ' We used the eleven components from the CP-FIT feedback cycle as the framework for interviews and analysis [4] . These are outlined in Table 2 , together with definitions and corresponding questions from the interview topic guide. Some interviews covered the entire cycle whilst others focused on specific components, depending upon participants' background and expertise. Interviews were completed by SW, lasted approximately 30-45 min, and were conducted remotely using telephone or video links. We first interviewed senior individuals from NDA and TARN to establish an understanding of how their programmes currently work, how their data are collected and processed (i.e. the 'audit' of audit and feedback), how their feedback is delivered to recipients, and clarify their recent innovations. These interviews helped us to develop short summaries of both audits that were provided to subsequent participants not already familiar with the audits (Additional file 2). We also sent participants an outline of CP-FIT, with linked interview questions, to familiarise them with the theory and help to focus attention on individual cycle components (Additional file 3). We invited participants to choose to discuss one or both audits. We initially used open-ended prompts to encourage discussion about features of the feedback cycle of particular interest. In later interviews, we ensured that specific cycle components were considered if they had not been covered in earlier interviews. Field notes were made immediately after each interview. Audio recordings were transcribed verbatim. We conducted a rapid, structured deductive content analysis of interviews exploring outputs from two national clinical audits. Our approach drew upon existing techniques [8, 9] and followed a two-stage process. We managed the data using Microsoft Excel. First, rather than line-by-line coding, individual transcripts were summarised using a one-page template that used the CP-FIT feedback cycle as a framework. Data were deductively added to the template to summarise participants' thoughts about how the cycle components applied to the audit(s). The template contained space for recording other issues that emerged inductively from the data and fell outside of the components specified in the CP-FIT feedback cycle. Most frequently, this concerned points raised in relation to particular clinical audits other than NDA or TARN, and therefore outside of the focus of our study. Templates were populated with key points and illustrative quotes. To ensure the reliability of interpretation, SW and TW independently prepared one-page summaries from the same two transcripts (one each focusing on NDA and TARN). These were checked to ensure that similar statements were generated within the different CP-FIT cycle components. Following discussion, two further transcripts were summarised and compared, with differences resolved by discussion. The remaining summaries were divided between SW and TW and completed individually, with regular meetings to check agreement. Examples of interview summaries are provided in Additional files 4 and 5. In the second stage, individual summaries were combined to create a matrix, one per audit, populated with summaries of participant comments that mapped to the cycle components. Each matrix was divided into two sections, the first including positive comments and the second detailing weaknesses and potentially failed components. The data for each audit (NDA and TARN) were considered separately; we did not combine data across audits. Data were analysed alongside CP-FIT's high-confidence hypotheses. Where data linked to a particular hypothesis, it was noted whether the associated audit programme had achieved the hypothesis or not. SW and TW met to identify and agree key strengths and weaknesses for each audit in relation to CP-FIT. We set ourselves a goal to identify three of each for each audit to ensure that we delivered practical and focussed outputs for our collaborating audit programmes. We shared our findings with senior managers at NDA and TARN whilst finalising the manuscript. We were particularly keen to ensure that the potential limitations identified for both audits were justified. Data were analysed in relation to all components of the feedback cycle, for both audits. However, to reduce unnecessary words and allow greater focus upon the most important findings, the study team identified which cycle components to present here, and which to include in Additional file 6. All authors contributed to this decision process. The study was approved by the University of Leeds School of Medicine Research Ethics Committee (ref: MREC 18-051) and the Health Research Authority (IRAS ID: 258139). We invited 25 individuals to participate. Nineteen responded before our recruitment deadline, and we completed eighteen interviews, one involving two Are the standards of clinical performance clear? Automated data collection and analysis processes are generally recommended. Manual collection and analysis are often hindered by a lack of time or skills. Who does the data collection? 3. Feedback Current best evidence supports more frequent provision of feedback. Data should also be as recent as possible, which may enhance subsequent cycle components (Acceptance, Intention, and behaviour) and encourage identification of suboptimal performance. Other relevant factors include problem-solving and action planning, i.e. helping recipients identify and introduce solutions to improve. What feedback is communicated? This component includes the method of delivery and how recipients interact with the feedback, e.g. is it delivered directly to clinicians or do they need to seek it out? How is the feedback received? Feedback is more effective when it is user-friendly. Provision of a comparator (e.g. showing performance benchmarked against appropriate others) is considered to facilitate the perception, Intention, and behaviour components. How is the feedback understood? Acceptance where, if the feedback permits, recipients can explore the data underlying performance. Can the recipients interrogate the data? 7. Acceptance Acceptance is facilitated when recipients believe the feedback presents a true representation of their performance. Users are more likely to engage with credible feedback, which facilitates several cycle components. Is there Acceptance of the feedback? Ideally, recipients form Intentions to take actions to improve performance in response to the feedback. Does the feedback elicit a planned response? Feedback that has been received, understood, and accepted will ideally be followed by a planned behavioural response. A distinction is made between patient-level responses, i.e. relating to the care of individuals, and those at the organisational-level with impacts across the wider healthcare system. Is the behavioural response at patient or organisation level? 10. Clinical performance improvement Organisation-level behaviours are associated with greater clinical performance improvement potential as they enable multiple patient-led behaviours by enhancing the clinical environment in which they occur. Are there positive changes to patient care as a result of feedback? 11. Unintended consequences CP-FIT acknowledges the potential for both positive and negative unintended outcomes of feedback interventions. Examples include improved recordkeeping, or manipulation of patient populations to artificially improve performance, respectively. Are there any unintended consequences as a result of the feedback? people, between June and August 2020. Eight participants were clinicians or managers, seven researchers, four patient and public representatives, and three audit providers; several interviewees had multiple roles (Table 3) . Table 4 presents an overview of the extent to which both audit programmes were considered to have achieved the CP-FIT feedback cycle. Cells indicating mixed features (+/-) indicate where interviewees identified both strengthening and limiting features of the feedback. We present our findings first for NDA, then TARN, focusing on the components considered most relevant when identifying three strengths and three areas for improvement for each audit. We made this pragmatic decision in setting ourselves goals to deliver key 'headline' measures for our national audit programme partners. The results for the remaining feedback cycle components are provided in Additional file 6. Data collection and analysis (Fig. 1, feedback Automated data collection places no additional demands on practice staff. This reduces the complexity [italics indicate CP-FIT explanatory mechanisms] of participation, increasing the likelihood of progress through the feedback cycle. The CP-FIT 'ownership' hypothesis is also However, more frequent feedback may cause alert fatigue (D18, audit and feedback researcher), and its impact depends on the number of patients included in the feedback, and the time taken for any change to occur. NDA feedback may often fail to reach its intended audience. Participants with primary care roles reported having little awareness of the feedback, which was not discussed by their practice teams. It had to compete for attention with other priorities: "As a recipient, I think that's probably been one of the key issues is that I've just not looked at it very much because it's not really been pushed in my face or kind of promoted" D16, audit and feedback researcher and GP This critical break in the feedback cycle means that feedback content cannot be received and acted upon. Interviewees expressed major reservations about the presentation of feedback data, variously described as very dense (D7, hospital consultant), overwhelming (D11, audit and feedback researcher), and really quite dysfunctional and unappealing (D9, hospital consultant). Participants consistently observed that the data were practically impenetrable on first look, with few users likely to have the time nor inclination to extract what was personally relevant. "[Quarterly feedback] is useful as long as it is deci-pherable data. If you are just sent a massive spreadsheet that you don't read, it doesn't really matter whether it is quarterly or annually, it's still not going to be read. " D7, hospital consultant One interviewee highlighted the cognitive effort required to act on the feedback: " [Recipients] have to do a lot of work in order to take their one line from this data…and turn it into useful information. …. Unless it's made dead easy, and people are interested and want to engage in it then they won't even make use of the easy to use displays, never mind the stuff where they actually have to go and do some work. " D18, audit and feedback researcher Another factor that hindered the interpretation of feedback was the lack of a comparator to help recipients understand their performance against others or over time. Whilst the NDA data release permits users to compare performance against others, it requires manipulation of a complex spreadsheet. "If I find that I'm 20% lower than where other people are just now that sets off alarm bells. … On the other hand, if I'm doing 20%, 30% better than expected, actually that is really useful to know… I can say, ok, so that's something I'm not going to worry about for the next two-three months. " D2, audit and feedback researcher and GP "The numbers alone are not enough. They need to be accompanied by specific targets so that progress towards these can be assessed. " D13, PPI representative Acceptance Participants believed that the feedback, if read, would be accepted by recipients. The NDA was considered credible, and the clinical content was recognised as important. One GP participant noted that there was no segmentation of the data. Therefore, suboptimal performance may be attributed to, for example, patient demographics rather than the functioning of the practice team. Consequently, progress through the cycle to Intention, behaviour, and performance improvement would be unlikely. These two components were typically discussed together. One interviewee believed that the feedback would prompt recipients to consider their performance and ask themselves how they might act in response: Participants indicated that the feedback could be enhanced by incorporating recommendations on how to improve, which is consistent with two CP-FIT hypotheses: that feedback is more effective when it helps recipients identify and develop solutions to reasons for suboptimal performance ['problem solving'], and provides solutions to suboptimal performance ['action planning']. Providing clear guidance would reduce the effort required of recipients in identifying solutions, thereby reducing feedback complexity and making it more actionable. One suggestion was to accompany the feedback with tools that support the creation of relevant patient lists so that they might be invited for review: The absence of recommendations may undermine any potential gains arising from more frequent data release: "It's not just a matter of having the data and having it good quality, and giving it to the practitioners, you have to do more than that in order to get them to use it. " D18, audit and feedback researcher We identified three main strengths of the NDA feedback that were consistent with CP-FIT and three opportunities for improvement (Table 5) . Data collection and analysis (Fig. 1, feedback TARN data are manually processed. This can be timeconsuming and requires expertise to ensure accuracy. Two participants with experience in other secondary care audit programmes identified manual data collection as a potential weakness. One explained that manual processes were associated with ongoing questions about whether all data were necessary, who completes the work, and how this is resourced. Conversely, manual processing may heighten feelings of 'ownership' . Participants perceived TARN data collection to be efficient and rigorous, but saw value in introducing automation: "We should be moving to a world in which the data is collected not essentially paper-based, manually, but is collected more remotely, electronically, effi-ciently… I think there's a lot of scope for doing that and I think that's something important. " D12, hospital consultant Descriptions of the TARN Analytics tool were generally positive, but this did not guarantee use. Recipients may take time to adjust to new systems, with some preferring More frequent data release This appears to meet user needs and is consistent with the best evidence about improving feedback effectiveness (Acceptance, Intention). Delivery to target recipients The feedback may be failing to reach much of its intended audience. If staff are unaware of the feedback then it cannot be discussed nor prompt improvement (Interaction). Automated data collection This minimises the impact upon the staff and helps to ensure a large, accurate dataset. It also reduces complexity and strengthens the initial processes of the cycle (Data collection and analysis). Those who do receive it may not read it: participants found it off-putting and impenetrable (Perception). The feedback is considered to come from a credible source; the indicators are recognised as relevant and important (Acceptance). Too much was required of users to produce useful comparator detail (Perception, Intention). "I guess this appeals to people like me and will appeal to medical directors…and it probably will be important in terms of influencing doctors perhaps and some nurses, but not everybody… there's definitely a thing for me about the difference in audiences. Whose performance you are actually trying to, whose behaviour are you trying to change?" D9, hospital consultant The TARN Analytics dashboard was considered attractive and user-friendly, facilitating progress through the feedback cycle. However, there remained scope for improving design and reducing complexity: Participants recognised that TARN Analytics enhanced users' ability to interrogate their data, strengthening the Verification component of the cycle. Those familiar with other audits described the ability to 'drill down' into data as something regularly requested by colleagues: "I think that's a good idea, I mean that's almost the comment that comes back on [our] audit is that sites want to drill down their own data, they want to understand their own practice in more detail. " D12, hospital consultant Providing optional further detail for those who wish to explore it is consistent with recommended practice. One participant referred to levels of interest and catering for different types of users: The participant working in trauma care suggested that the new tool would have time-saving benefits. He described an occasion when achievement on an indicator had suddenly and unexpectedly declined. The staff spent a long time investigating the issue and what might explain the change. He believed that TARN Analytics would simplify such processes: "I would hope it could help pinpoint faster that it is, e.g. two outliers that have caused it, or is it one It was apparent that TARN feedback was respected and considered credible by both its target audience and clinicians from other specialties. The trauma network manager (D5) confirmed that the data were an integral part of care at all levels of his organisation. Data were used by teams to assess the impact of quality improvement projects and featured in peer review visits and local executive meetings. Participants again noted that it was not made clear what was required of recipients in response to feedback. The trauma network manager (D5) suggested that action was required primarily of clinicians: "For me, the target audience is less people like me and more actually clinical, clinicians and surgeons and consultants. " He appeared to consider himself more as an intermediary who could extract detail from the feedback and share it with others, e.g. directors and clinicians. It was not clear whether or how such individuals would then act. One participant discussed how feedback should be targeted, such that specific recommendations are congruent with recipients' organisational role. For example, if highlevel actions are required, then any recommendations need to be for tasks that appropriate individuals or teams can actually deliver. Moreover, recipients at those levels must also understand that action is required of them. We again identified three strengths of the feedback and three opportunities for improvement as a practical output for our audit partners ( Table 6 ). The strengths facilitate achievement of the cycle components and encourage progress through the cycle. Opportunities for improvement concern issues most likely to inhibit progression. The results suggest relatively smooth progress around the feedback cycle up to Intention (component 7, Fig. 1) , becoming more uncertain beyond that point. Opportunities for improvement The TARN Analytics tool was considered a useful innovation. It aligns with recommended practice, allowing users to 'drill down' into the data (Perception, Verification). Action planning Providing recommended actions to guide recipients on how to improve would make the feedback more actionable (Intention, Behaviour). Use of comparator Performance presented as relative to the national average, stimulating social influence (Perception, Intention). Comparator flexibility Offering users the ability to select preferred comparator(s) could be more beneficial than a standard national average (Perception, Intention). Respected source Feedback considered to come from a credible source. TARN recognised as an exemplar from which other audits could learn (Acceptance) Evidence of impact Clearer demonstration of impact upon patient care would likely strengthen the feedback cycle (all). This is the first study to apply a comprehensive, healthcare-specific theory of feedback (CP-FIT) [4] to assess the clinical audit programmes. Changes introduced by two audit programmes: increased frequency of feedback by the NDA and an analytics tool allowing interrogation and verification of feedback data by TARN, were consistent with best practices according to CP-FIT and augmented existing strengths. We also identified potential weaknesses in feedback cycles, such as the omission of specific actions plans for improvement and limited demonstration of audit impacts on patient care and outcomes. We have previously demonstrated variations in how feedback is delivered by UK national clinical audit programmes [13] . This closer analysis and comparison of two audit programmes has highlighted a range of opportunities to improve impacts specific to each programme. Participants identified elements from both audits that were likely to facilitate progress through the CP-FIT feedback cycle and make it more likely that the audit programme would contribute to improved clinical performance. The NDA was described as an established, clinically relevant initiative, with efficient and widely respected data collection and analysis processes. The move to more frequent data release was consistent with the best evidence and consolidated existing programme strengths. TARN was recognised as a respected and proactive audit programme, including by those from other secondary care specialities. The new TARN Analytics tool was described as an attractive and engaging feature and being of particular benefit to those users seeking to explore their data in more detail (augmenting the Perception, Verification, and Acceptance components of the feedback cycle). Our findings highlight the importance of paying attention to all aspects of the feedback cycle to optimise impact. The feedback cycle is perhaps only as strong as its weakest link; any breakdown at one or more points in the cycle undermines the ability of an audit programme to drive improvement. For example, releasing feedback data more frequently is unlikely to enhance effectiveness if feedback has a limited or unknown reach and is difficult to comprehend. The potential limitations we observed in relation to NDA and TARN are consistent with the wider audit and feedback literature. For example, ensuring that feedback is user-friendly and enables recipients to efficiently obtain the information they need (i.e. reducing cognitive load and feedback complexity) is considered a key requirement of an effective feedback intervention [4, 6] . User-centred design processes may help to ensure userfriendly feedback designs by identifying and rectifying problems earlier in the design process [14] . Similarly, feedback that also supports action planning or presents specific recommendations for action is consistently recommended by reviews in this field [2, 4, 6] . Ivers et al. [15] observed that recipients of an audit and feedback intervention to improve chronic disease management struggled to use it as a means to set specific improvement goals, highlighting the necessity of supporting this process as part of the intervention. CP-FIT hypothesises that offering solutions helps to facilitate the Intention and behaviour components of the feedback cycle. This is because providing solutions makes it simpler for health professionals to undertake each cycle component (i.e. reduces complexity) and makes it easier for them to act in response to the feedback (i.e. increases actionability). Issues around the selection of comparators were identified in relation to both audits considered here. For the users of TARN, performance is routinely presented in comparison with the national average, whereas one interviewee commented that this might not be the most effective approach. A review of the comparators utilised in 146 randomised trials of audit and feedback [16] concluded that theory and evidence were underused in deciding which comparator(s) to include. One recommendation was that the comparator be tailored to individual recipients rather than benchmarking against the average as standard. This point was raised by our interviewee who described the importance of considering suboptimal performance from a local context and not the national perspective. It is important to optimise major audit programmes, given upfront investments in establishing infrastructures and involving staff in data collection and the potential for far-reaching, population-level benefits [1] . A repeated analysis of UK national clinical audit reports identified trends towards the use of more effective feedback methods (e.g. greater use of recommendations for action) and identified further opportunities for improvement [13] . CP-FIT offers a rigorously derived framework for evaluating feedback interventions and explaining their observed or predicted effects [4] . Our rapid, frameworkdriven analysis demonstrates an approach to identify the strengths and limitations of audit programmes, and hence guide their further development. We acknowledge the four main study limitations. First, our rapid evaluation, albeit as a necessary response to the coronavirus pandemic, had less rigour and depth compared to other qualitative methods. Nevertheless, rapid approaches to qualitative analysis are gaining recognition as an acceptable methodology within limited timeframes and have been successfully applied to other contexts [10] . Comparative studies have demonstrated that rapid analysis techniques can generate similar findings to indepth approaches [8, 9, 11] . In particular, they can allow researchers to work productively with partners, such as audit programmes, that often seek timely and actionable recommendations. Second, pandemic restrictions prevented the recruitment of actual feedback recipients as originally intended. The people we interviewed about TARN included three clinicians already familiar with the NDA but only a single individual with direct experience of TARN. Consequently, this individual's perceptions feature prominently, although balanced with those of other secondary care clinicians. A related limitation is that we deliberately approached several expert participants with interests in audit and feedback whose perspectives will differ from more typical feedback recipients. Repeating this work with a sample of non-expert, end-users would offer the opportunity to compare and contrast the present findings. Third, we did not formally assess for data saturation and cannot make claims to saturation of themes. However, our sample comprised 19 individuals offering a variety of practice, patient, and research perspectives on feedback. Time limitations prevented us from adding interviews beyond this point. Fourth, our assessments considered the design and delivery of audit programmes in relation to theoretical best practices. However, what is theoretically recommended may not translate into real benefits when effectiveness is empirically evaluated [17] . We acknowledge the need to continue building the evidence base for audit and feedback [18] . Our findings have direct implications for both evaluated audit programmes: we have identified theorybased issues for both that may limit the effectiveness of both audits and their ability to improve patient care. Moreover, it is likely that the strengths and improvement opportunities identified are common to other national clinical audits. For example, difficulty in reaching those most able to respond effectively to feedback has been identified as a challenge for another national audit programme [19] . Audit programmes need to work through how to identify or develop their recipient communities and ensure that feedback is designed with consideration of their informational needs and capacity for effective action. We also noted a relative paucity of specific recommendations on how to improve performance. The Cochrane review of audit and feedback found that feedback was more effective when accompanied by clear targets and action plans [2] . Further research may examine relationships between key features of audit and feedback interventions, as delineated by CP-FIT, and their effects on clinical practice and outcomes. A further finding of wider relevance to audit programmes is the need to demonstrate their impact, both to improve recipient engagement and justify continued programme funding. Our findings may guide the external review and commissioning of national audit programmes. We have identified features of feedback from two national audit programmes likely to enhance and impede progress around the feedback cycle. For this commonly used improvement method, our findings suggest considerable scope to improve impacts on service delivery and patient care through these and other audit programmes. Those responsible for commissioning and delivering major audit programmes should encourage the systematic identification of key opportunities for improvement across the whole audit and feedback cycle. Revitalising audit and feedback to improve patient care Audit and feedback: effects on professional practice and healthcare outcomes A systematic review of the use of theory in randomized controlled trials of audit and feedback Clinical Performance Feedback Intervention Theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research National Diabetes Audit Programme Practice feedback interventions: 15 suggestions for optimizing effectiveness The Trauma Audit & Research Network Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis Rapid techniques in qualitative research: a critical review of the literature Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR) Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups Repeated analyses of national clinical audit reports demonstrate improvements in feedback methods Designing clinical practice feedback reports: three steps illustrated in Veterans Health Affairs long-term care facilities and programs My approach to this job is...one person at a time": perceived discordance between population-level quality targets and patient-centred care. Can Fam Physician Clinical performance comparators in audit and feedback: a review of theory and evidence The OFF theory of research utilization No more 'business as usual' with audit and feedback interventions: towards an agenda for a reinvigorated intervention Can we do better? Bridging the research to practice gap in patient blood management-optimizing 'audit & feedback' and the challenges of undertaking a national cluster-randomized controlled trial Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations Not applicable. The online version contains supplementary material available at https:// doi. org/ 10. 1186/ s43058-022-00275-5. Additional file 2. Short summaries of NDA and TARN shared with participants prior to interview.Additional file 3. CP-FIT feedback cycle and questions for discussion. Example of an interview summary (NDA). Additional file 6. Results for the remaining CP-FIT feedback cycle components.Authors' contributions TW drafted the paper. SW completed the interviews. TW and SW completed the analysis. SW, JB, HC, BB, FL, and RF commented on drafts. RF was the principal investigator. The authors read and approved the final manuscript. The datasets used during the current study are available from the corresponding author on reasonable request.