key: cord-0980711-m4kayc7t authors: Hill, Nathan R.; Arden, Chris; Beresford-Hulme, Lee; Camm, A. John; Clifton, David; Davies, D. Wyn; Farooqui, Usman; Gordon, Jason; Groves, Lara; Hurst, Michael; Lawton, Sarah; Lister, Steven; Mallen, Christian; Martin, Anne-Celine; McEwan, Phil; Pollock, Kevin G.; Rogers, Jennifer; Sandler, Belinda; Sugrue, Daniel M.; Cohen, Alexander T. title: Identification of undiagnosed atrial fibrillation patients using a machine learning risk prediction algorithm and diagnostic testing (PULsE-AI): Study protocol for a randomised controlled trial date: 2020-10-19 journal: Contemp Clin Trials DOI: 10.1016/j.cct.2020.106191 sha: 22051445ead1b981922e2509beb55e184e7d60c3 doc_id: 980711 cord_uid: m4kayc7t Atrial fibrillation (AF) is associated with an increased risk of stroke, enhanced stroke severity, and other comorbidities. However, AF is often asymptomatic, and frequently remains undiagnosed until complications occur. Current screening approaches for AF lack either cost-effectiveness or diagnostic sensitivity; thus, there is interest in tools that could be used for population screening. An AF risk prediction algorithm, developed using machine learning from a UK dataset of 2,994,837 patients, was found to be more effective than existing models at identifying patients at risk of AF. Therefore, the aim of the trial is to assess the effectiveness of this risk prediction algorithm combined with diagnostic testing for the identification of AF in a real-world primary care setting. Eligible participants (aged ≥30 years and without an existing AF diagnosis) registered at participating UK general practices will be randomised into intervention and control arms. Intervention arm participants identified at highest risk of developing AF (algorithm risk score ≥ 7.4%) will be invited for a 12‑lead electrocardiogram (ECG) followed by two-weeks of home-based ECG monitoring with a KardiaMobile device. Control arm participants will be used for comparison and will be managed routinely. The primary outcome is the number of AF diagnoses in the intervention arm compared with the control arm during the research window. If the trial is successful, there is potential for the risk prediction algorithm to be implemented throughout primary care for narrowing the population considered at highest risk for AF who could benefit from more intensive screening for AF. Trial Registration: NCT04045639 Atrial Fibrillation (AF)the most common sustained heart arrhythmia 1is associated with a five-fold increase in stroke risk 2 and enhanced stroke severity compared to patients without AF, resulting in greater disability and higher mortality 3, 4 . As AF can be intermittent and asymptomatic it is challenging to diagnose. Data indicate that only 70-85% of patients living with AF have been formally diagnosed 5, 6 . Early identification of undiagnosed AF and appropriate management including anticoagulation therapy is essential to reduce stroke risk in patients with AF. Multiple screening approaches are available for the identification of AF, all with varying cost-effectiveness and diagnostic ability 7 . Screening tends to be opportunistic (e.g. screening of patients attending their general practitioner (GP) for another reason), targeted (e.g. screening of higher-risk patients), or systematic (e.g. screening all patients aged >65 years), and can include simple pulse checking or more resource-intensive electrocardiogram assessment (ECG). European guidelines recommend opportunistic screening via pulse taking or ECG rhythm strip in patients ≥65 years of age, and only recommend systematic ECG screening in individuals ≥75 years or those at high risk of stroke 7 . Opportunistic screening is recommended because it is more likely to be cost-effective than systematic population screening approaches [7] [8] [9] . However, opportunistic screening relies on patients visiting their GP for a different reason and pulse checking often lacks diagnostic precision 7,10 , resulting in many patients with intermittent AF having a missed diagnosis, and others undergoing unnecessary further testing. These limitations, alongside further uncertainties about the benefits of screening and the effectiveness of treatment for AF in patients identified through screening are likely reasons for the absence of a national screening programme for AF in the UK 11 . Therefore, given the J o u r n a l P r e -p r o o f Journal Pre-proof estimated size of the undiagnosed AF population (approximately 300,000 individuals in the UK 5, 13 ) , and the absence of cost-effective systematic screening approaches for AF, there is interest in methods that are cost-effective, diagnostically precise, and able to be applied across a population to narrow the patient population at highest risk of AF that should undergo more intensive screening for AF. Existing risk prediction models for AF include the CHARGE-AF 14 14 ) , the machine learning algorithm was more effective at identifying patients with AF, reducing the number of patients needed to screen to identify one case of AF (with 75% sensitivity) from 13 to 9. However, whether this risk prediction algorithm when combined with diagnostic testing is accurate at identifying undiagnosed AF in a real-world primary care setting is currently unknown. Therefore, the Prediction of Undiagnosed atriaL fibrillation using a machinE learning AlgorIthm (PULsE-AI) randomised controlled trial is designed to assess the real-world ability of the machine learning AF risk prediction algorithm coupled with diagnostic testing (ECG +/-KardiaMobile) for the identification of patients with AF compared to routine care. J o u r n a l P r e -p r o o f The primary objective of the PULsE-AI trial is to assess the effectiveness of an AF risk prediction algorithm and diagnostic testing (ECG +/-KardiaMobile) for the identification of patients with AF in primary care, compared to routine care. The primary outcome/endpoint is the prevalence of confirmed AF (% with diagnosed AF) during the research window in intervention and control arms. The secondary objective is to assess the economic impact (healthcare resource utilisation, life years, and quality-adjusted life years) of using the AF risk prediction algorithm and diagnostic testing (ECG +/-KardiaMobile) for the identification of patients with AF. The PULsE-AI trial is a prospective, randomised controlled trial conducted in the UK primary care setting. Eligible adult patients registered at participating general practices within the NIHR Clinical Research Network: West Midlands will be identified from medical records and then individually randomised to receive routine care plus the intervention/screening strategy (intervention arm) or routine care only (control arm). Diagnostic data will be extracted from medical records of both intervention and control arm participants at the beginning and end of the research window to determine the proportion of patients diagnosed with AF throughout the trial in the intervention and control arms. The research window is defined as the time between the first collection of patient medical records at the beginning of the trial (06 June 2019) and the last collection following the completion of the intervention at the end of the trial. This research window will not be fixed but will be sufficient to allow participating practices enough time to conduct the intervention. The research window will be consistent across sites and trial arms to ensure the same time period for AF diagnosis remains the same for all participants. This protocol has been designed in accordance with the Standardized Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines and checklist (Additional file 1). A schedule for enrolment, intervention and assessment (SPIRIT figure) is outlined in Table 1 . A CONsolidated Standards of Reporting Trials (CONSORT) diagram is shown in Figure 1 . Adult patients (≥ 30 years) without a diagnosis of AF prior to 06 June 2019 will be identified from the medical records of participating general practices. Patients aged ≥ 30 years were included because this age group were included in the development of the machine learning risk-prediction algorithm. Patients will be eligible for participation in the trial if they have a complete set of key clinical measurements (height, weight, body mass index (BMI), systolic blood pressure (SBP), and diastolic blood pressure (DBP); i.e. a valid index date) recorded during a rolling 12-month window ending at any time during the 11 years prior to the start of the research window). All eligible patients will then be randomised into intervention and control arms in a 1:1 ratio via simple randomisation. Intervention and control group patients with a risk score ≥ 7.4% will form the primary analysis population, and all randomised patients will be included in a sensitivity analysis. Following randomisation and AF risk score generation, general practitioners (GPs) will review the lists of patients randomised to the intervention arm at highest risk of AF before letters of invitation are sent, to ensure all patients are suitable to participate in the trial. GPs may choose not to invite patients based on their clinical judgment (e.g. patients receiving end of life care, patients with active cancer, patients whose mental health may be negatively affected by being invited to participate in the J o u r n a l P r e -p r o o f Journal Pre-proof trial) and the rationale for exclusion will be captured. The response rate among invited higher risk intervention participants is expected to be ~30%. The AF risk prediction algorithm was developed using machine learning techniques and data from a retrospective cohort of 2,994,837 adult patients (aged ≥ 30 years) without a history of AF (prior to January 2006) and listed on the CPRD GOLD between January 2006 and December 2016 18 . During the study period, 3.2% of the cohort were diagnosed with AF. Both baseline (patient demographics (age, sex, race, smoking status), history of antihypertensive use, type 1 or type 2 diabetes, and cardiovascular comorbidities) and time-varying (recent cardiovascular event(s), recent BMI and change in BMI, recent pulse pressure, change in SBP and DBP, and recent frequency of SBP, DBP, and BMI recordings) patient data were incorporated into the machine learning algorithm to generate a risk score for AF, with the three most important predictors of AF found to be recent heart failure, coronary heart disease, and myocardial infarction 18 (Supplementary Figure S1 ). The algorithm was calibrated 19 during development, using cross-validation to prevent overfitting and random sampling to define training and holdout sets, ensuring the final set was broadly representative of what the model was trained on. The risk prediction algorithm will be run on data from all eligible patients with a valid index date following randomisation, and a risk score will be generated for each individual. A risk score of ≥ 7.4% was chosen as the threshold needed for the AF risk prediction algorithm as this corresponded to a sensitivity of 50% (Supplementary Table S1 ), as assumed in the sample size calculations. Intervention group participants with a risk score ≥ 7.4% and suitable for participation in the study will be invited to attend a research clinic for diagnostic testing. Following informed J o u r n a l P r e -p r o o f Journal Pre-proof consent, participant characteristics and clinical history will be recorded by the research nurse, and a 12-lead ECG performed. Intervention arm participants not diagnosed with AF following the 12-lead ECG will be provided with a KardiaMobile device (AliveCor Inc., California, USA), if they have access to a compatible smartphone or tablet. Those without access to a compatible smartphone or tablet will be offered a loan smartphone to use alongside the KardiaMobile for the study period and, if declined, will be invited for up to two further 12-lead ECGs. Participants using the KardiaMobile will be asked to record their ECG twice daily (morning and evening), in addition to any time they feel unwell, for two weeks. Participants will be diagnosed with AF by a cardiologist if they have ≥ 30 seconds of arrhythmia, typical of AF, as per standard guidelines 12, 20 . In addition, participants who are unable to attend the research clinic (e.g. because they are housebound or do not have available transport) will be offered a visit at home with a portable ECG, followed by the two weeks of KardiaMobile monitoring described above. All intervention arm participants with a cardiologist-confirmed diagnosis of AF (either from the 12-lead ECG or KardiaMobile data) will have met the primary endpoint for the study and will be evaluated for anticoagulation therapy as per local procedures. Intervention arm participants may also be diagnosed with AF through routine clinical practice. In addition, participants diagnosed with other cardiac abnormalities will also be referred and treated according to local policies. Control arm participants will have no direct contact with investigators during the study. Control arm participants will only be diagnosed with AF through routine clinical practice, and any diagnoses will be made according to local policies. J o u r n a l P r e -p r o o f Owing to the pragmatic design of the trial, the recruitment of six study sites will result in an estimated 24,000 patients who meet the inclusion criteria. Based on data from the AF risk prediction algorithm development 18 , it is assumed that approximately 1,000 patients per arm will be at higher risk of AF (risk score ≥ 7.4%). Of these 1,000 intervention arm participants, it is estimated that: ~30% will accept the invitation to attend the research clinic (see Supplementary Figure S2 for a summary of estimated patient and participant numbers in each cohort of the trial); 1 in 16 of those who attend the clinic will be diagnosed with AF; andover a six month period -0.7% of those who do not attend the research clinic will be diagnosed with AF 18 . Thus, assuming an AF diagnosis rate of 2.4% in higher risk patients in the intervention arm and 0.7% in the control arm, and a 5% Type-I error rate, this yields a statistical power for the study of 88.5%. Power calculations were conducted using the 'pow' package in R version 3.4.2. Demographics and clinical characteristics for all randomised participants, and for the primary analysis group of higher risk participants (risk score of ≥ 7.4%), will be summarised using descriptive statistics. If the baseline characteristics of the intervention and control arms are well balanced, the proportion of patients diagnosed with AF during the research window will be compared between the intervention and control arms via an unadjusted chi-squared comparison. Alternately, if there are baseline differences between arms, these will be adjusted for within a penalised logistic regression. A sensitivity analysis will also be performed on all participants randomised into intervention and control groups irrespective of predicted AF risk score. All statistical tests will be conducted at the 5% significance level. The primary outcome will be utilised to inform an economic analysis designed to evaluate the health economic impact of a screening strategy using the AF risk prediction algorithm J o u r n a l P r e -p r o o f (combined with a diagnostic test(s)) for the identification of patients with undiagnosed AF (secondary objective). The cost-effectiveness model will compare the use of the AF risk prediction algorithm intervention as an add-on to current practice against current practice alone. The model will use AF incidence rates determined from this trial alongside data from published literature (for variables such as AF-related event rates and cost estimates) to evaluate outcomes over a patient's lifetime. The economic value of the AF risk prediction algorithm will be captured through the estimation of total costs, life years gained as a result of earlier diagnosis of AF, and quality-adjusted life years gained as a result of avoiding AFrelated events due to earlier diagnosis. In line with ethical approval requirements, pseudonymised patient-level data from participating study sites will be extracted from the Egton Medical Information System (EMIS) medical records of all eligible patients via a search and report function. Eligible patients without a valid index date will be excluded prior to the randomisation of patients at each study site. Following randomisation, data will be used to generate AF risk scores for all participants in the intervention and control arms, and those at highest risk of AF (risk score ≥ 7.4%) identified. Study sites will then de-anonymise records of the highest risk patients in the intervention arm for invitation. Once participants have signed informed consent, data collected during and after the research clinic appointment will be entered into the EMIS electronic medical records for each participant. All ECG data will be pseudonymised for cardiologist review. All KardiaMobile data will be uploaded to secure European Servers based on unique study identifiers. Pseudonymised data will be shared with the trial sponsor (and its delegates) using secure file transfer protocol (sFTP), which will ensure an end to end encryption of data extracted. Three years of additional follow-up data may be collected from J o u r n a l P r e -p r o o f the medical records of participants identified as higher risk by the AF risk prediction algorithm to evaluate the impact of the intervention on clinical outcomes, but will not involve direct patient contact. Consent for all pseudonymised data sharing outside of that collected at the research clinic has been provided by the general practice participating in the study. Ethical approval for this study was granted on 28 February 2019 (Wales Research Ethics The PULsE-AI trial is designed to assess the performance of a machine learning AF risk prediction algorithm alongside diagnostic testing (12-lead ECG +/-KardiaMobile) at identifying and detecting patients with undiagnosed AF, and thus evaluate a complete screening strategy that could be applied in a current UK primary care setting. Whilst the AF risk prediction algorithm assessed in this study has been shown to reduce the potential number of patients needed to screen to detect undiagnosed AF by 31% 18 following theoretical validation, whether it is similarly effective at identifying patients at higher risk of AF within a real-world primary care setting is currently unknown. Current AF screening approaches are limited by either cost-effectiveness or diagnostic precision 8, 12 . Emerging technologies such as the KardiaMobile monitor and smartwatches are now widely available and increase the accessibility of home-based ECG recording; however, they can be expensive and may be technologically prohibitive to older patient populations who are at higher risk of AF. Indeed, the recent Apple Heart Study 21 invited people with access to a smartwatch to participate in a study. The study sought to measure the proportion of participants with an irregular pulse notificationand confirm AF or atrial flutter with an ECG. However, this study was skewed in favour of young to middle age adults who were most likely to have the technology, reducing the applicability of findings to older populations at highest risk of AF. Machine learning techniques appear to be effective in the identification of undiagnosed AF [22] [23] [24] , however all known studies involved the application of machine learning algorithms to ECG data. The novelty of this study is that the AF risk prediction algorithm does not require ECG data as an input and utilises routinely collected data contained within existing medical records. If successful, this approach which does not rely on existing ECG data or on patient access to emerging technologies could be of significant value to patients, clinicians, and budget-constrained healthcare systems alike. The success of the study depends on several factors. First, the assumptions made during the sample size and power analysis calculations are correct. Calculations have been made based on observations from the AF risk prediction algorithm study 18 , and yield a statistical power of 88%. Any changes in these assumptions could potentially affect the number of AF diagnoses made during the research window and therefore the primary outcome. Second, all calculations were made based on the population distribution as per the CPRD GOLD dataset used to develop the risk prediction algorithm. Therefore, if local populations vary significantly from the CPRD GOLD dataset (e.g. they are younger), more practices may need J o u r n a l P r e -p r o o f to be recruited to reach a sample size of approximately 1,000 higher risk participants per arm. Third, AF will be diagnosed during the study via 12-lead ECG or the KardiaMobile device. Whilst participants without a personal smartphone or tablet (or access to a device via a family member) will be offered a loan smartphone for the KardiaMobile monitoring period, some may lack confidence with technology and either decline the offer or are unable to use it correctly. Although these participants will be invited for subsequent 12-lead ECGs, the absence of home-based ECG monitoring may reduce the ability to confirm the actual presence or absence of AF. This trial will be subject to several limitations. As with all multicentre studies, the generalisability of trial findings to patients/settings beyond the participating practices and surrounding areas is uncertain. Therefore, generalisations will be made with caution and both known and unknown differences in the study population that may impact the applicability of the trial will be discussed. Due to the nature of the trial, patients may be 'flagged' as high risk of AF by the algorithm but are not subsequently diagnosed with AF. However, given the often-intermittent nature of AF, and/or the absence of a compatible smartphone in some participants, it will not be possible to confirm a genuine absence of AF in this subset of patients. Furthermore, it is possible that patients identified by the algorithm as higher-risk may have been picked up later by routine methods (lead-time bias). However, as both the intervention arm and the control arm have the same chance to identify AF patients opportunistically, we should be able to quantify this bias. Due to the individually randomised design, both intervention and control participants will be present at study sites. It is, therefore, possible that physicians may increase their efforts to J o u r n a l P r e -p r o o f diagnose patients beyond what would be considered routine care as a result of being involved in an AF study (performance bias). To ensure the control arm closely reflected routine clinical care, we only recruited practices that were not active in cardiovascular-related research at the time of trial site recruitment. Practices were not restricted from participating in any local, regional or national AF detection initiatives during the course of the trial. Every effort will be made to ensure that patients in the control arm and patients in the intervention arm will receive routine care as is usually delivered in clinical practice through appropriate training of practice researchers. Additionally, participants will be informed not to allow family or friends to use their KardiaMobile device as it could artificially inflate the number of diagnoses in either the control or intervention arms. Finally, the COVID-19 pandemic may impact on trial recruitment because of the suspension of all non-essential services in primary care in England. Whether this will have an impact on the trial beyond simply extending the trial period is currently unknown. Timely diagnosis of AF and optimal anticoagulation therapy (where required) can reduce the risk of AF-related events such as stroke. If the PULsE-AI trial is successful, the algorithm could become a valuable diagnostic adjunct, implementable in primary care for narrowing the population considered at highest risk of AF who should undergo more intensive testing (e.g. with a 12-lead ECG), and therefore enable a more cost-effective strategy of screening for undiagnosed AF. Currently, six general practices are enrolled in the study. Patients at these practices have been randomised, and those at highest risk of AF identified. The first participant attended the Participating General Practices have already provided consent on behalf of their patients for pseudonymised data to be used for research. Informed consent will be obtained for all participants randomised to the intervention group who accept the invitation for diagnostic testing before the collection of any data from the research clinic appointment. -T1: activities prior to randomisation/treatment allocation, including determination of patient eligibility from medical records and generation of an AF risk score; T0: randomisation/treatment allocation; T1: research clinic appointment; T2 completion of KardiaMobile monitoring period; T3: follow-up to assess the number of AF diagnoses in intervention arm participants beyond the research window. * If unsuitable for KardiaMobile monitoring (i.e. without access to a compatible smartphone/tablet) † Data on related events (e.g. stroke, mortality etc.) may be collected alongside AF diagnoses. J o u r n a l P r e -p r o o f Worldwide epidemiology of atrial fibrillation: a Global Burden of Disease Atrial fibrillation as an independent risk factor for stroke: the Framingham Study Characteristics, outcome, and care of stroke associated with atrial fibrillation in Europe: data from a multicenter multinational hospitalbased registry (The European Community Stroke Project) Contribution of atrial fibrillation to incidence and outcome of ischemic stroke: results from a population-based study Atrial fibrillation prevalence estimates in England: Application of recent population estimates of AF in Sweden AHSN Network atrial fibrillation programme 2020 ESC Guidelines for the diagnosis and management of atrial fibrillation developed in collaboration with the European Association of Cardio-Thoracic Surgery (EACTS) A randomised controlled trial and cost-effectiveness study of systematic screening (targeted and total population screening) versus routine practice for the detection of atrial fibrillation in people aged 65 and over. The SAFE study Screening strategies for atrial fibrillation: a systematic review and cost-effectiveness analysis Accuracy of methods for detecting an irregular pulse and suspected atrial fibrillation: a systematic review and meta-analysis UK National Screening Committee. Evidence Summary for Screening for Atrial Fibrillation in Adults: External review against programme appraisal criteria for the UK National Screening Committee ESC Guidelines for the management of atrial fibrillation developed in collaboration with EACTS Atrial fibrillation: finding the missing 300,000 Simple risk model predicts incidence of atrial fibrillation in a racially and geographically diverse population: the CHARGE-AF consortium A clinical risk score for atrial fibrillation in a biracial prospective cohort (from the Atherosclerosis Development of a risk score for atrial fibrillation (Framingham Heart Study): a community-based cohort study. The Lancet Selective screening for atrial fibrillation using multivariable risk models Predicting atrial fibrillation in primary care using machine learning Calibration: the Achilles heel of predictive analytics Guidelines for the management of atrial fibrillation: the Task Force for the Management of Atrial Fibrillation of the European Society of Cardiology (ESC) Large-Scale Assessment of a Smartwatch to Identify Atrial Fibrillation Identification of patients with atrial fibrillation: a big data exploratory analysis of the UK Biobank An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction Diagnostic assessment of a deep learning system for detecting atrial fibrillation in pulse waveforms Medical writing and editorial support in the preparation of this manuscript were provided by Ltd. Logistical support for the study will be provided by the School of Primary, Community and Social Care, Keele University, Staffordshire, UK.J o u r n a l P r e -p r o o f