key: cord-0772647-wftd6hyb
authors: Hafiz, Nashid; Hyun, Karice; Tu, Qiang; Knight, Andrew; Hespe, Charlotte; Chow, Clara K.; Briffa, Tom; Gallagher, Robyn; Reid, Christopher M.; Hare, David L.; Zwar, Nicholas; Woodward, Mark; Jan, Stephen; Atkins, Emily R.; Laba, Tracey-Lea; Halcomb, Elizabeth; Johnson, Tracey; Usherwood, Timothy; Redfern, Julie
title: Data-driven quality improvement program to prevent hospitalisation and improve care of people living with coronary heart disease: Protocol for a process evaluation
date: 2022-05-17
journal: Contemp Clin Trials
DOI: 10.1016/j.cct.2022.106794
sha: 94680f3474d3af298afe84ed26ae7c9f24c956cb
doc_id: 772647
cord_uid: wftd6hyb

Background Practice-level quality improvement initiatives using rapidly advancing technology offers a multidimensional approach to reduce cardiovascular disease burden. For the “QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with heart disease” (QUEL) cluster randomised controlled trial, a 12-month quality improvement intervention was designed for primary care practices to use data and implement progressive changes using “Plan, Do, Study, Act” cycles within their practices with training in a series of interactive workshops. This protocol aims to describe the systematic methods to conduct a process evaluation of the data-driven intervention within the QUEL study. Methods A mixed-method approach will be used to conduct the evaluation. Quantitative data collected throughout the intervention period, via surveys and intervention materials, will be used to (1) identify the key elements of the intervention and how, for whom and in what context it was effective; (2) determine if the intervention is delivered as intended; and (3) describe practice engagement, commitment and capacity associated with various intervention components. Qualitative data, collected via semi-structured interviews and open-ended questions, will be used to gather in-depth understanding of the (1) satisfaction, utility, barriers and enablers; (2) acceptability, uptake and feasibility, and (3) effect of the COVID-19 pandemic on the implementation of the intervention. Conclusion Findings from the evaluation will provide new knowledge on the implementation of a complex, multi-component intervention at practice-level using their own electronic patient data to enhance secondary prevention of cardiovascular disease. Trial registration: Australian New Zealand Clinical Trials Registry (ANZCTR) number ACTRN12619001790134.

people surviving initial cardiac events, the prevalence of CVD is increasing along with the economic cost [7] . To reduce the risk of future cardiovascular events in those with established disease, secondary prevention strategies have become an international priority [8, 9] and include the use of guideline-indicated medications, adopting a healthy lifestyle, implementation of chronic disease management plans and participation in a cardiac rehabilitation program following an acute event [8] . Primary care plays an integral role in implementing successful secondary prevention strategies as the majority of people hospitalised for CVD regularly visit their primary care practitioners and use governmentfunded health services at least once a year following their acute CHD diagnosis [10] [11] [12] [13] .

Funded by the federal government, primary care is the first point of contact for all Australians to access care. Under the primary care system, individuals can receive services that includes treatment of acute conditions, chronic disease management, health promotion, prevention and early intervention [14] . These services are provided via the general practices, community health centres and allied health practices with the help of Primary Health Networks (PHNs) [14, 15] . The Australian government has recently launched the Quality Improvement Practice Incentive Program (QI-PIP), which encourages primary care practices to collaborate with their PHNs and undertake quality improvement activities within their practices to provide high-quality patient care for better health outcomes [16] . As a result, many primary care practices worldwide are rapidly adopting the use of quality improvement (QI) initiatives [17- conditions including asthma [23] , diabetes, neonatal health [24, 25] . However, there is a paucity of research focused on evaluating the effectiveness of such QI interventions only at individual patient level rather than community or clinic level in CVD management. [26, 27] .

The -QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with heart disease‖ (QUEL) study aimed to determine the effectiveness of a quality improvement program for improving CVD management [28] .

For the QUEL cluster randomised controlled trial (cRCT), a structured QI program is delivered within Australian primary care practices to reduce CVD hospitalisations, improve CVD risk factors and medication adherence in patients with CHD over 24 months [28] . The intervention practices are supported by the study team or their relevant PHNs to enhance efficiency in management and outcomes of CVD patients by better using their routinely collected data. The trial is ongoing involving 52 (27 intervention and 25 control) Australian primary care practices with approximately 15,000 CVD patients with 12-and 24-month follow-up with data collection scheduled for completion in mid-2022. The primary outcome is CVD hospitalisations, collected via linkage with state-based administrative data linkage centres that collect data on all hospitalisations in Australian hospitals and as such will not be adjudicated, and secondary outcomes are cardiovascular risk factors recorded electronically by the GPs in real time which will be collected routinely across all participating primary care practices using a standardised data extraction software; medication prescriptions and use

The QUEL intervention is based on the Collaborative Framework [29] and consists of (1) a virtual orientation session, (2) electronic data collection at baseline, and thereafter monthly, from the intervention practices via a practice-level software system that enables automated data extraction [30], (3) monthly data reporting, (4) completion of Plan, Do, Study, Act (PDSA) cycles that summarise practice-level progress towards pre-determined CVD indicators for the QUEL study, (5) a series of interactive learning and benchmarking workshops (1 in-person and 5 virtual sessions due to the COVID-19 pandemic) and (6) provision of support from PHNs and the study team. The intervention is delivered over a period of 12 months.

The study team is collaborating with five PHNs to ensure optimal delivery of the collaborative intervention. PHNs are independent organisations funded by the Australian government aimed to coordinate health services for the communities in a specific region [15] .

PHNs also work closely with the primary care practices and other health care professionals within the region to identify gaps and build capacity to ensure optimal service delivery [15].

There are thirty-one PHNs operating in Australia including in remote and Aboriginal Torres Strait Islanders communities to encourage use of available health resources and access health care [15] . All PHNs were invited to participate through a variety of communication channels, including a mailing list direct to PHN CEOs, University of Sydney's as well as research partners' networks. Five out of the thirty-one PHNs agreed to collaborate on the study based in ensuring successful implementation of the intervention. The role of the PHN involves, but is not limited to, supporting practices to achieve pre-defined key performance measures to optimise outcomes, participating in program activity including training and learning workshops, encouraging practice level engagement in these activities and using PDSA cycles between activity periods, sharing practice achievement and providing additional support as required.

Process evaluation is particularly important in complex intervention trials as it provides indepth information required to evaluate the intervention's effectiveness and investigate the implementation process. It provides valuable insights into describing the various intervention components [31] and identifying factors associated with successes and challenges of the programs in various healthcare settings [32] . Use of process evaluation alongside complex interventions is increasing given because of the associated multisite, multicomponent features [33, 34] . However, little research has reported the mechanisms of impact, context and what constitutes effective QI interventions aimed at improving CVD management in primary care settings.

The QUEL QI program is a complex intervention with multiple interactive components, as such, process evaluation can accurately describe the intervention implementation, exposure of the intended intervention and real-time experiences of those involved [35] . We hypothesise that evaluating the implementation of the multi-component QI intervention within the QUEL trial will help primary care practices to undertake further QI activities to improve care of CVD within their practices. The earlier protocol describes the cluster RCT itself [28] , while this current protocol details the evaluation plan for the data-driven QI intervention program within the QUEL cRCT and its effects. The process evaluation aims to: J o u r n a l P r e -p r o o f

Explore to what extent the intervention is delivered as intended, identify key elements of the intervention associated with positive study outcomes, and how, for whom and in what context it was effective.

Describe and analyse practice engagement, attendance, time commitment, software capability, skills and capacity of the practice team members associated with attending learning workshops.

Understand acceptability, satisfaction, uptake, utility and feasibility of the QI program.

Identify and describe barriers and enablers of the QI program.

Evaluate the effect of COVID -19 on the implementation of the QI program.

A mixed-methods approach will be undertaken using data from 27 intervention practices (out of 52 participating practices) from the QUEL cRCT [36, 37] . For this study, data will be collected only from the intervention practices as it aims to evaluate the effect of the QI intervention program. Qualitative and quantitative data will be collected both during and at the end of the trial intervention period. Semi-structured interviews and open-ended questions will be used to collect qualitative data. Quantitative data will be collected from the intervention practices via multiple data sources throughout the intervention period.

A program logic model was developed to describe how, why and among whom the collaborative intervention works in practices within the QUEL cRCT ( Figure 1 ). This logic model is a visual representation of the intervention design and its intended implementation.

The Cochrane Effective Practice and Organisation of Care Review Group (EPOC) checklist was used as a guidance to develop the logic model [38] to identify the key features of the J o u r n a l P r e -p r o o f Journal Pre-proof intervention, check the fidelity of the implementation and assess participant's experience [31] . The model includes 5 domains of the intervention: (1) input, (2) activities, (3) outputs, (4) outcome and (5) impact specific to the data-driven QI program that will be used to describe the study objectives [39] . Inputs refer to various resources that are required to ensure program operation, activities refer to the planned actions, such as delivery of workshops, data collection that are an essential part of the implementation [39] . Resources, inputs, and activities together form the program design. Outputs include the changes in the participant's behaviour, knowledge, skills, and awareness resulting from the activities and impact describes the fundamental changes occurring in the health services over a longer period as a result of the program activities [40] .

Participants in the process evaluation will include practice team members (including general practitioners, nurses and practice managers) from primary care practices allocated to the intervention arm and PHN staff who are providing direct support to the intervention practices under their jurisdiction. At least two practice team members from all 27 intervention practices who were actively involved in QI activities in their practices; such as participated in QI workshops, submitted and carried out PDSA cycles and regularly communicated and shared reports with the study team on their activities during the intervention period will be approached to complete the surveys and participate in semi-structured interviews. These participants will be able to understand sufficient English to provide written and informed consent. Practice team members from primary care practices allocated to the control group and any PHN staff not involved in the QUEL project will not be included in the process evaluation. All practice team members who are part of the intervention practices will be J o u r n a l P r e -p r o o f Journal Pre-proof approached for recruitment to provide feedback and participate in interviews during and at the end of the intervention.

Multiple data sources, collected throughout the cRCT, will be used in addition to surveys and interview data, to evaluate whether the complex intervention was delivered as planned.

Combining these data sources will help to identify the key intervention elements, identify the dose, frequency and activities delivered to the intervention practices as well as describe barriers and enablers associated with the program implementation.

To maintain balanced quality of information across the multiple data sources the research team will ensure a) close communications and interaction with practices and PHNs (e.g. workshops; practice visits, regular contact via email or phone calls) to promote quality of data collection; b) an experienced research officer is responsible for data collection throughout the study; c) all the practices are well informed on the study procedures before they are enrolled; d) the participating practices receive appropriate research support when required; e) routine extraction, monitor and check data for quality assurance and help practices solve issues if data is not returned. The data sources will include: 1) practice-level enrolment data, 2) attendance record, 3) SharePoint data, 4) practice correspondence record, 5) data collection record, 6) PDSA cycles, 7) learning workshop surveys, 8) end of program survey and 9)

semi-structured interviews of practice team members and PHN representatives. These data sources will be used as credible evidence collected at different time point during the intervention period ( Figure 1 ).

Practice-level enrolment data will be created at the time of recruitment and will be recorded in a Microsoft Excel (2016) spreadsheet. Information collected will include practice location (urban and rural), practice team members information, software compatibility, and randomisation group. Urban and rural primary care practices were defined using the Australian department of Health's Health workforce classification guideline [41] . This spreadsheet will be used throughout the main trial period and updated regularly with current dates and version numbers. We will use these data to identify intervention practices, software eligibility and installation requirement of the eligible software, practice support, describe practice type and gather details of the practice team members involved in the delivery of the intervention.

Participation of the practice team members in any events related to the intervention including orientation and learning workshops (both face-to-face and virtual) will be recorded in another

Microsoft Excel (2016) spreadsheet and updated regularly throughout the intervention period.

These data will provide information on the frequency of the workshops attended by the intervention practices, the number of staff from each practice attending the orientation and workshops.

Microsoft SharePoint [42] is an online platform where a unique account is created for individual practices in both intervention and control arms. This platform is created for the practices to submit their PDSA cycle records and track improvements via monthly graphs which are uploaded in their respective accounts by the study team. From the SharePoint data, we will identify whether each practice had access to their account and all intervention 

The study team will be communicating with the intervention practices during the trial period and practices will also be encouraged to directly communicate with the study team as required. These communications will be undertaken via phone call, email or in-person site visits. Any communication will be saved and used to identify the reason, mode of contact (email, site visit or phone call), time spent on the contact, person contacted and solution provided in a Microsoft Excel (2016) spreadsheet. This document will be updated throughout the study period with current dates and version numbers.

The intervention practices will submit clinical data electronically in an aggregated and deidentified form monthly via the automated data extraction software [30]. All aggregated data will be stored in the University's Research Data Storage (RDS). These data will be used to create practice level reports and will be uploaded to practices' SharePoint sites monthly as graphs for benchmarking their improvement for the pre-defined QUEL study performance measures. A Microsoft Excel (2016) spreadsheet will be used to record monthly data collection and reporting for each intervention practice. This spreadsheet will also be updated regularly throughout the intervention period with current dates and version numbers.

The PDSA cycle is a simple but powerful tool to measure improvements and increasingly used in many QI collaborative to boost quality of healthcare [43, 44] . It guides users to J o u r n a l P r e -p r o o f Journal Pre-proof explicitly plan, implement, reflect on, and then repeat, incremental improvements as they make system changes to achieve the aim [45] . Practices participating in the QUEL QI intervention are required to document and upload their PDSA cycles using a template.

Training will be provided to the practices during the learning workshops on the process of completing PDSAs. Submitted PDSA cycles will be saved in their respective SharePoint accounts and the study team will be able to download a copy of the cycle when required. We will use all the PDSA cycles submitted by primary care practices during the intervention period to gather information on practice engagement, number of PDSAs submitted by each practice, identify key areas practices focused on improving and identify barriers and enablers to make improvement changes within the practices.

At least two practice team members (one clinical and one administrative) will be invited to participate in a series of six learning workshops that will be delivered during the intervention period. Six surveys corresponding to six workshops will be administered at the end of each recorded with consent and verbatim method will be used for transcription of the interviews.

Interview transcript, any free text and notes from the interviews as well as the surveys and PDSAs will be coded and managed in NVivo Software.

All data collected for the process evaluation including personal information will be securely stored in the University's RDS database. Access to the RDS will require an employee unikey and password and only a limited number of people will have access to it. All data will be stored on The University's RDS for the duration necessary to comply with regulatory requirements; thereafter database will be destroyed in accordance with University's IT recommendations. Completed surveys and interview data will be stored securely for 5 years after publications, after which time, they will be destroyed securely. No personal information will be published. At the end of this data retention period, all files will be physically destroyed.

This protocol outlines the systematic methods of a process evaluation of a complex QI intervention embedded within a cRCT to improve secondary prevention of CVD in primary care. The evaluation logic model is described along with methods for understanding the impact of the intervention and the context in which the impact occurs. It will assess successes and failures related to the program implementation in addition to determining factors associated with program scale-up and adaptation for other primary care settings [32] . The EPOC framework does not provide information on describing the actual QI intervention, therefore this evaluation also includes interviewing of key participants at completion of the intervention. The study will contribute to stronger evidence around the use of QI in primary J o u r n a l P r e -p r o o f Journal Pre-proof care to improve CVD outcomes as well as to literature through encouraging the development of process evaluation methodology in the design and promoting transparency in the reporting of the findings.

A strength of our study is the use of a mixed-methods approach. Mixed-methods research can strengthen data quality, improve interpretations of findings, and offer a more comprehensive understanding of the program implementation, and hence, it has become a very useful tool to evaluate complex interventions [51] . Quantitative data will provide key information on what was effective and qualitative data will provide deeper understanding of why and in what context the intervention was effective. Therefore, combination of both will provide a more holistic understanding of the complex intervention than either method alone [52] .

Furthermore, this approach enables a richer perspective from a range of participants (GPs, nurses, practice managers, and PHN representatives) using various surveys and semistructured interviews integrated within the main QUEL cRCT. Combining interview and survey data will enable in-depth knowledge on program utility, barriers, and likelihood of adoption. Findings from the process evaluation will also inform other primary care practices to implement data-driven QI programs and provide valuable insights to policymakers on wider adoption and scaling-up of such strategies.

While this process evaluation will enable evaluation of a complex QI intervention and barriers and enablers to its implementation, there are several limitations. One of the limitations is the data collected from the interviews may be subjected to recall bias as interviews will take place after the intervention. Another limitations is the cRCT is designed to be delivered in Australian primary care environment, hence it may only be relevant to health systems with similar contexts, funding, and infrastructures.

Global burden of CVD: focus on secondary prevention of cardiovascular disease

World Health Organization. Cardiovascular diseases CVDs Fact Sheet

World Heart Federation. CARDIOVASCULAR DISEASE: THE WORLD'S NUMBER 1 KILLER. INFOGRAPHICS. 2021

Economic Costs of Cardiovascular Diseases in Poland Estimates for 2015-2017 Years

World Heart Federation. Champion Advocates Programme: The costs of CVD

Costs of Cardiac Care Likely to Increase, Despite Advances in Prevention, Care

Secondary prevention of coronary heart disease in Australia: a blueprint for reform

ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: The Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC)

Transition between hospital and community care for patients with coronary heart disease: New South Wales and Victoria 2012-2015. Australian Institute of Health and Welfare

Regular primary care plays a significant role in secondary prevention of ischemic heart disease in a Western Australian cohort

The self-perceived role of general practitioners in care of patients with cardiovascular diseases. A survey in Central and Eastern European countries following health care reforms

Person-centred care after acute coronary syndrome, from hospital to primary care-a randomised controlled trial

Department of Health. PIP QI Incentive guidance Canberra

Use of quality improvement strategies among small to medium-size us primary care practices

A quality improvement collaborative to build improvement capacity in regional primary care support organisations

Batalden PB, Davidoff F. What is -quality improvement‖ and how can it transform healthcare?

What is "quality improvement" and how can it transform healthcare? Qual Saf Health Care

Are quality improvement collaboratives effective? A systematic review

The foundation of improvement. Quality Progress

Does quality of care affect rates of hospitalization for childhood asthma? Pediatrics. meta-analysis. The Lancet

Evidence for the impact of quality improvement collaboratives: systematic review

A realist evaluation approach to explaining the role of context in the impact of a complex eHealth intervention for improving prevention of cardiovascular disease

What drives adoption of a computerised, multifaceted quality improvement intervention for cardiovascular disease management in primary healthcare settings? A mixed methods analysis using normalisation process theory

QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with coronary heart disease (QUEL): protocol for a 24-month cluster randomised controlled trial in primary care

The improvement guide: A practical approach to enhancing organizational performance

Process evaluation of the data-driven quality improvement in primary care (DQIP) trial: active and less active ingredients of a multi

Improving care at scale: process evaluation of a multi-component quality improvement intervention to reduce mortality after emergency abdominal surgery (EPOCH trial)

Protocol for a process evaluation of a cluster randomized controlled trial of the Learning Club intervention for women's health, and infant's health and development in rural Vietnam

Protocol for a process evaluation of a cluster randomised controlled trial to improve management of multimorbidity in general practice: the 3D study

Process evaluation on quality improvement interventions

Research design: Qualitative, quantitative, and mixed methods approaches

Designing and conducting mixed methods research

Cochrane Effective Practice and Organisation of Care Review Group (EPOC). Data collection checklist

The logic model guidebook: Better strategies for great results

WK Kellogg Foundation logic model development guide. WK Kellogg Foundation

Use the PDSA model for effective change management. Education for Primary Care

A primer on PDSA: executing plan-do-study-act cycles in practice, not just in name

Redesigning the care of rheumatic diseases at the practice and system levels. Part 1: practice level process improvement (Redesign 101)

Qualitative research method-interviewing and observation

Qualitative methods in research on healthcare quality

The qualitative research interview

Doing a thematic analysis: A practical, step-by-step guide for learning and teaching scholars

Learning to do qualitative data analysis: A starting point

Introduction to mixed methods in impact evaluation

Figure 1: Logic model for data-driven Quality improvement (QUEL) intervention process evaluation CVD: Cardiovascular Disease, CHD: Coronary Heart Disease, PEN CS: Pen Computer Systems, GP: General Practitioner, PDSA: Plan, Do, Study. Act, PHC: Primary Health Care, IT; Information Technology

The authors acknowledge the support of all the PHN and primary care practices who continue to support the QUEL project. Also, Pen CS for providing the services and eHealth data platform for the study; and the Improvement Foundation for their continuous support in the delivery of the QI program and other study partners including; Inala Primary Care, Fairfield

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Figure 1