key: cord-0052007-9o1cinds
authors: Davidson, Jennifer; Banerjee, Amitava; Muzambi, Rutendo; Smeeth, Liam; Warren-Gash, Charlotte
title: Validity of Acute Cardiovascular Outcome Diagnoses Recorded in European Electronic Health Records: A Systematic Review
date: 2020-10-14
journal: Clin Epidemiol
DOI: 10.2147/clep.s265619
sha: 2e879fe0bf73198d26a3ec87bf558bf33a1f8699
doc_id: 52007
cord_uid: 9o1cinds

BACKGROUND: Electronic health records are widely used in cardiovascular disease research. We appraised the validity of stroke, acute coronary syndrome and heart failure diagnoses in studies conducted using European electronic health records. METHODS: Using a prespecified strategy, we systematically searched seven databases from dates of inception to April 2019. Two reviewers independently completed study selection, followed by partial parallel data extraction and risk of bias assessment. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value estimates were narratively synthesized and heterogeneity between sensitivity and PPV estimates were assessed using I(2). RESULTS: We identified 81 studies, of which 20 validated heart failure diagnoses, 31 validated acute coronary syndrome diagnoses with 29 specifically recording estimates for myocardial infarction, and 41 validated stroke diagnoses. Few studies reported specificity or negative predictive value estimates. Sensitivity was ≤66% in all but one heart failure study, ≥80% for 91% of myocardial infarction studies, and ≥70% for 73% of stroke studies. PPV was ≥80% in 74% of heart failure, 88% of myocardial infarction, and 70% of stroke studies. PPV by stroke subtype was variable, at ≥80% for 80% of ischaemic stroke but only 44% of haemorrhagic stroke. There was considerable heterogeneity (I(2) >75%) between sensitivity and PPV estimates for all diagnoses. CONCLUSION: Overall, European electronic health record stroke, acute coronary syndrome and heart failure diagnoses are accurate for use in research, although validity estimates for heart failure and individual stroke subtypes were lower. Where possible, researchers should validate data before use or carefully interpret the results of previous validation studies for their own study purposes.

Ischaemic heart disease and cerebrovascular disease have been the leading causes of death globally for more than 15 years. 1 In Europe, cardiovascular disease (CVD) deaths and prevalence have decreased but remain substantial; in 2015 an estimated 85 million people had CVD including 11.3 million with new diagnoses. 2 CVD determinants and outcomes research increasingly utilize electronic health records (EHRs). EHRs contain comprehensive longitudinal health data, extracted from primary and secondary care clinical systems, for large patient populations which provide cost-effective data for research. EHR data is mostly "structured" with diagnoses coded using, for example, the International Classification of Diseases (ICD) but can also be "unstructured" with anonymized free-text notes. 3 EHR-based research predominantly uses structured data. As the primary purpose of EHR data collection is clinical, it is essential to consider the validity of the data's use in research.

EHR use is widespread in Europe, where many countries have national healthcare systems, and several systematic reviews have previously explored the quality of specific European EHRs. [4] [5] [6] [7] Other systematic reviews [8] [9] [10] [11] [12] have investigated the validity of CVD diagnoses in computerized health-related records, which included EHRs but mainly drew results from disparate claims-based systems. The previous reviews did not separate results for EHR and claims data, the quality of which may differ due to the differences in setup and collection rationale.

In our systematic review, we provide an up-to-date assessment of the validity of acute CVD diagnoses recorded in European EHRs. We defined acute CVD as heart failure (HF), acute coronary syndrome (ACS), and stroke. These high-burden conditions are key diagnoses commonly included in the composite endpoint of major adverse cardiovascular events (MACE) which is increasingly employed in both clinical trials and observational research studies. 13 We investigated whether the validity of these diagnoses differed by subtype, definition, data source, reference standard, and study population.

Our protocol was published in October 2019 14 following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol guidelines (PROSPERO registration number CRD42019123898).

We included articles that validated diagnoses in patients aged ≥16 years captured in any European primary or secondary care EHR. We excluded claims-based databases, disease registries, vital registration systems, or locally held databases. Articles needed to validate clinical codes for the diagnoses of HF, ACS, or stroke (Table 1) against a suitable internal or external reference standard. HF is most frequently a chronic condition which can deteriorate with acute exacerbations. HF may also have an acute onset, for example after an MI. The European Society of Cardiology (ESC) defines acute HF as rapid onset or worsening of symptoms and/or signs of existing HF. 15 ACS encompasses different clinical forms of myocardial ischaemia which includes myocardial infarction (MI) and unstable angina. The specific diagnosis of MI or unstable angina depends on symptoms, signs, biomarkers, and ECG and/or autopsy findings, with the definitions refined over time. 16 The diagnosis of stroke includes subtypes ischaemic stroke, intracerebral haemorrhage (ICH), and subarachnoid haemorrhage (SAH). 17 At least one validation estimate (Figure 1 ) or the raw data to calculate it was required.

We searched for eligible articles in five databases (Medline, Embase, Scopus, Web of Science, and Cochrane Library), two grey literature sources (OpenGrey and Ethos), and, where available, the bibliographies of EHR databases from the date of inception to April 2019 in any language.

We searched medical subject heading terms and free-text (in the title and abstract) for the concepts of (1) CVD 

Two reviewers (J.A.D. and R.M.) independently screened the titles and abstracts of all retrieved articles, followed by the full-text of articles deemed eligible in the first stage. Our published protocol details the full data collection process. 14 Briefly, we extracted data using a pre-defined template (S2 Appendix) which we piloted using dual extraction for three studies, followed by further parallel extraction for 20% of studies, and completed by a single reviewer (J.A.D.) for the remaining studies.

We used a modified version of the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) 18 tool to assess bias (S3 Appendix). As with our data extraction, two authors (J.A.D. and R.M.) piloted the tool for three studies, then independently assessed risk in a further 10% of studies, with the process completed by a single reviewer (J.A.D.).

We synthesized results with a narrative approach, grouping studies by acute CVD diagnosis (HF, ACS or stroke) and, where possible, subgroups of interest. Subgroups were; diagnosis type, definition, data source including diagnostic position and coding system, reference standard, and study population including time period, age and sex. For studies that reported validation estimates without confidence intervals (CIs), but included raw data, we calculated 95% CIs using the Wilson method for binomial proportions. We used the I 2 statistic to assess heterogeneity between the sensitivity and positive predictive value (PPV) estimates, following the Cochrane thresholds. 19 Heterogeneity assessment did not include specificity or negative predictive value (NPV), as few studies reported these measures.

To investigate sources of heterogeneity, we compared I 2 before and after removing studies at a high risk of bias and by the previously mentioned subgroups. We used the Stata metaprop command 20 to calculate I 2 . Metaprop uses raw data rather than precalculated estimates; studies that reported sensitivity or PPV but not the data used to calculate were excluded from heterogeneity assessment.

We used the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) tool for diagnostic accuracy systematic reviews 21 to summarise crossstudy quality. Evidence was categorised as "high", "moderate", "low" or "very low" quality. See S4 Appendix for the reasons we rated quality down or up.

We identified 4595 studies, of which 218 were included in full-text review and 81 met eligibility criteria ( Figure 2 ). Study characteristics are summarized in S1 

The 81 included studies validated EHRs from 11 different countries, most frequently Denmark (18 studies) [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] and the UK (17 studies). [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] Validation was the primary aim of all but 10 studies. 35, 36, 41, 48, [57] [58] [59] [60] [61] [62] Fourteen studies 26, 27, 31, [63] [64] [65] [66] [67] [68] [69] [70] [71] [72] [73] validated a vital registration system or disease registry in 27, 30, [35] [36] [37] 46, 49, 52, 55, 57, 58, 61, 66, 74, [81] [82] [83] [84] 95 and low for eight (10%) of studies. 40, 41, 45, 48, 71, 80, 91, 97 Studies were overall at low risk of bias in patient selection (76 low, 3 unclear, 2 high), index test (71 low, 10 high), and flow and timing (78 low, 3 unclear) domains and higher risk in the reference standard domain (36 low, 28 unclear, 17 high). Generally, reference standard methods and definitions were poorly described, and on occasion the reference standard was not independent of the EHR. Risk of bias was also higher in studies which validated primary care EHRs. HF validation studies had high quality in 14 (70%) studies, medium in five (25%) and low in one (5%). For ACS validation, quality was high for 21 (68%), medium for eight (26%) and low for two (6%) studies. In stroke validation studies, quality was high for 26 (63%), medium for nine (22%) and low for six (15%) studies.

HF diagnoses were most extensively validated using EHR data from Denmark (five studies), 24, [28] [29] [30] 33 the Netherlands (four studies), 59, 65, 94, 95 Sweden (three studies) 82,83,88 and the UK (three studies). 43, 46, 54 In addition, EHR data from Finland, 67 France, 77 Germany, 85 46, 82, [94] [95] [96] [97] Medical record review was used as the reference standard in all but three studies. 24, 65, 97 Heart Failure Validation Results

From the main validation result reported by each of the studies; sensitivity (available from nine studies)- 24, 46, 65, 67, 77, 82, 85, 88, 95 was ≥50% in six studies 46, 77, 82, 85, 88, 95 but >66% (range 11-100%) in only one study, 46 PPV (19 

In the three studies that reported results for first diagnosis, the PPV range was 76-88%. 28, 29, 77 One study compared the PPV for all diagnoses (84%) to first diagnosis (80%), 28 and another study found the same PPV for first diagnosis and recurrent diagnosis (both 76%). 29 Definition In seven of the eight studies 24 

Six studies 29, 33, 43, 54, 77, 83 reported HF recorded in any diagnostic position (PPV 76-96%) and two studies 30, 88 only included primary position (PPV 87% and 100%). Three studies, 33, 77, 83 which validated any position, also included breakdowns by primary (PPV 88-96%) and secondary (PPV 66-84%) positions. 46, 52 Four studies 22, 37, 68, 76 presented overall ACS results, of which one study 68 included an additional breakdown for MI and two studies 37,76 included unstable angina and MI, one of which also included cardiac arrest. 37 A further two studies 29,65 did not report results for ACS overall but did include both unstable angina and MI. The remaining 25 studies solely validated MI diagnoses. 23 

The PPV for unstable angina varied; with low values of 20% 76 and 27.5% 37 in two studies and higher values of 78% 65 and 88% 29 in the other two studies. Sensitivity was only reported by one study, 65 at 53%. For MI, the main validation result for sensitivity (11 studies) 26, 27, 34, 42, 46, 50, 58, 65, 81, 88, 98 was ≥80% in all but one study 42 (range 56-97%), and six 26, 27, 34, 58, 88, 98 >90%. PPV (24 studies) 23, 26, 27, 29, 30, 32, 34, 37, 42, 46, 47, 50, 52, 53, 58, 65, 70, 75, 76, 80, 84, 87, 88, 98 was ≥80% (range 42-100%) in all but three studies 27,32,34 with 12 23, 29, 30, 42, 50, 52, 53, 65, 87, 88, 98 ≥90%. Three studies 34, 42, 98 reported specificity (range 93-100%) and two- 34, 98 included NPV (range 82-100%).

Four studies 29, 32, 37, 84 reported the PPV for first MI, with estimates of 75-97%, and one study 29 also included recurrent MI with a PPV of 88% compared to 97% for first MI.

Varying MI definitions were used (S6 Appendix). Most frequently (nine studies) 26, 27, 50, 70, 75, 81, 84, 99, 100 the World Health Organization (WHO) Monitoring trends and determinants in cardiovascular disease (MONICA) definition 106 was used, with variable PPV estimates of 53-96% obtained. Two studies compared MONICA to another MI definition; one 75 showed MONICA-defined definite MI had a substantially lower PPV than AHA/ESC-defined 16 definite MI (53% vs 86%), while the other 84 also showed a lower PPV for MONICA compared to "normal clinically defined MI" but with a smaller difference (81% vs 89%). One further study used the AHA/ESC definition 37 (PPV 82%). The universal definition 107 was used in a study 23 which included EHR data from three countries, with PPVs of 75-100%. Three studies used the third universal definition, 108 one 76 of which combined it with the earlier universal definition (PPV 85%). In another 53 PPVs of 92% with obtained for the primary and secondary care EHRs validated. The third 34 validated MI diagnoses recorded for patients with drug-eluting coronary stents, the PPV was 42% for all admission and 73% for acute admissions.

Of the 10 studies which reported the diagnostic position used to validate MI diagnoses, five 26, 27, 29, 34, 68 used any diagnostic position (PPV 42-97%) and five 30, 75, 76, 88, 98 primary position (PPV 53-100%). One study 27 which 

Ten studies validated ICD-10 coded MI, eight reported results specifically for ICD-10. 23, 29, 30, 32, 34, 47, 53, 76 Four studies validated ICD-10 I21 with PPV ≥85% (range 42-100%) 23, 29, 34, 76 in all but one. 34 Two studies included I21-I23 and reported high PPVs of 92% 53 and 98%; 30 however, the latter study was small in size (50 patients).

One study validated I21-I22 (PPV 89%) 47 and another I21-I24 (PPV 75%). 32 The estimates for ICD-10 codes were no higher than those for ICD-8 (PPV 79-100%), 26 Of the studies to validate data in primary care, one 23 included IPCI K75 code (PPV 75%) and three 50,52,53 validated Read coding in the UK (PPV 91-93%).

The PPV for MI diagnoses varied between 53-100% when medical record review was the reference standard (20 studies) 22 studies. 31, 35, [39] [40] [41] 45, 48, 49, 60, 63, 64, 71, 72, 81, 86, 87, 92, 98 Nine of the studies 31, 32, 40, 49, 60, 63, 64, 71, 101 did not include codes to validate SAH, three of which had stated this in their inclusion criteria. 40, 71, 101 Excluding these studies did not affect the sensitivity (53-89%) or PPV (68-97%). Specificity and NPV, reported by five studies, were 99-100% 49, 56, 63, 98 other than one study 31 which obtained a specificity of 96% and NPV of 72%.

Three studies 56, 64, 101 included first and recurrent overall stroke with sensitivity from 71-89% and PPV 69-81%, while three studies 32,71,73 also included only first stroke for which sensitivity was 85-89% and PPV 70-97%.

For ischaemic stroke, the main sensitivity reported (6 studies) 74, 79, 81, 86, 90, 102 was ≥66% in all but one 86 study (range 37-82%). Fourteen studies 25, 32, 38, 47, 57, 62, 72, 74, 78, 79, 86, 90, 92, 102 included one main PPV of 66-96%. One study 101 classified results separately for cardiac embolism, large artery atherosclerosis, lacunar infarct and ischaemic stroke of other aetiology. Sensitivity and PPV were highest in the cardiac embolism classification (83% and 87%, respectively) and lowest for other aetiology (67% and 35%, respectively). For ICH, the main sensitivity reported was 59-98% (4 studies) 73, 86, 101, 102 and main PPV 55-96% (15 studies). 25, 38, 39, 47, 51, 55, 57, 72, 73, 78, 86, 87, 92, 101, 102 The sensitivity of SAH diagnoses was 35-92% (4 studies)- 73, 81, 86, 102 and PPV was 42-96% (18 studies). 25, 36, 38, 39, 47, 51, 55, 61, 72, 73, 78, 81, 86, 87, 89, 92, 93, 102 Definition Stroke was defined in 22 of the 41 studies, [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] 31, 35, 38, 39, 63, 66, 71, 81, 86, 90, 92, 101, 102 

For overall stroke diagnoses recorded in any diagnostic positions, sensitivity ranged from 53-97% 56, 63, 86 and PPV from 69-90%. 25, 56, 63, 86 In comparison, results only for primary position were 67-86% for sensitivity and 69-95% for PPV. 49, 63, 73, 98, 101 Coding System 45 additionally included I62 (PPV 96%). The estimates for ICD-10 codes were no higher than those for ICD-8 codes (sensitivity 82%), 81 ICD-9 (PPV 20-95%, 40 73 Seven studies validated ICD-10 I63 for ischaemic stroke diagnosis (PPV 78-96%). 25, 32, 38, 47, 78, 79, 92 One study 73 used a broad (ICD-9433, 434, 436 and ICD-10 I63, I64) and narrow range of codes (ICD-9433, 434 and ICD-10 I63) to define ischaemic stroke, with similar sensitivity (82% vs 81%) and PPV (84% vs 83%). One other study 74 reported results by ICD-9 codes 443*1 and 434*1 (PPV 86% and 90%, respectively). Six studies 25, 38, 55, 78, 89, 92 validated ICD-10 I61, with another two 39, 101 presumed to have also validated this code, for ICH (PPV 66-96%) and a further three 

In the 17 studies 25, 31, 32, 35, 38, 39, 45, 55, 56, 60, 63, 72, 79, 86, 87, 91, 92 which used medical record review as the reference standard to validate overall stroke diagnoses, the PPV was ≥79% (range 20-97%) in all but four studies. 25 

We were able to assess the heterogeneity between the main PPV reported in; 14 studies with 16 estimates of HF (I 2 =97.0%), 18 studies with 26 estimates of MI (I 2 =98.5%), and 19 studies with 20 estimates of stroke (I 2 =97.9%) diagnoses. Additionally, we assessed heterogeneity between the main sensitivity for; six studies of HF (I 2 =98.6%), four of MI (I 2 =74.3%), and 11 of stroke (I 2 =98.8%) diagnoses. Heterogeneity between the estimates was considerable, at more than >95% in all cases other than sensitivity estimates for MI. Furthermore, heterogeneity remained considerable after removal of studies at a high risk of bias.

GRADE showed that cross-study quality was very low for all HF outcomes (sensitivity and PPV in secondary care EHRs and PPV in primary care EHRs), low for MI sensitivity and PPV in secondary care EHRs and moderate for PPV in primary care EHRs, and very low for stroke sensitivity in secondary care EHRs and PPV in primary care EHRs and moderate for PPV in secondary care EHRs.

Our systematic review suggests that the sensitivity of coded data in European EHRs for HF diagnoses is low at ≤66% in all but one study. There was also wide variation in stroke sensitivity estimates, with only half of studies ≥80%, although three-quarters were ≥70%. The sensitivity of ACS was higher at ≥80% in the vast majority of studies. The majority of studies which validated ACS diagnosis did so specifically for MI.

The PPV of all diagnoses was ≥80% in the majority of studies; two-thirds for HF (nearly three-quarters for secondary care EHRs), nearly three-quarters for MI, and 70% of stroke validation studies. Where subtypes were validated, PPV was ≥80% for four-fifths of ischaemic stroke diagnoses but only 44% of ICH and SAH diagnoses.

The specificity and NPV were also high where available (three HF studies, three MI studies and five stroke studies). However, as most studies only included patients with the diagnosis of interest recorded in the EHR and reference standard, the results presented were mostly limited to sensitivity and PPV.

Both PPV and NPV are impacted by disease prevalence, with lower estimates for rare conditions. 111 Our systematic review focused on Europe, drawing studies from 11 countries. Age-standardized prevalence of CVD in these countries is between 5000-6500 per 100,000, other than the Czech Republic (~8700 per 100,000) which only contributed one study. 2 Therefore, prevalence differences should have limited impact on our comparison of validity estimates between geographies. The prevalence of CVD increases with age, but we did not find any systematic difference in results between studies with younger or older populations.

The low sensitivity of HF diagnoses we identified is consistent with a previous systematic review validating HF diagnoses in administrative data, which identified three European studies. 11 Twelve more studies have since been published and included in our review. These more recent findings, however, do not suggest any improvement in the quality of data over time. This is perhaps unsurprising given the range of clinical aetiology and presentation. The high proportion of studies we found to have a PPV of <80% for stroke diagnoses appeared more substantial than in previous systematic reviews. 9, 12 We identified 15 new studies which were not included in these previous reviews. 25, 32, 45, 51, 56, 57, [61] [62] [63] 74, 78, 89, 91, 92, 98 Our results for sensitivity and PPV of MI diagnoses are consistent with previous reviews, 8, 10 and identified five 29, 32, 34, 76, 98 new MI validation studies with variable results.

There was substantial heterogeneity between the sensitivity and PPV estimates for all three acute CVD diagnoses. Heterogeneity was likely because studies differed in multiple ways; for example, even among studies which used medical record review as the reference standard, differences in study time period impacted upon the ICD version used. The heterogeneity caused by variable methods was highlighted in previous systematic reviews of 

We were most interested in the results of ICD-10 validation, as this is the latest ICD coding system which is widely used in Europe and elsewhere. In McCormick et al's 10 review of MI diagnoses in administrative data, the authors noted a lack of ICD-10 validation with only three studies identified, whereas our review identified 10. Nevertheless, even within ICD-10, combinations of codes used, and therefore their validity, differed, which highlights the importance of tailoring codes to each research question. Codes are arguably even more important when using other, more complex coding systems such as Read codes, which are used in UK primary care data and can generate vast numbers of codes for every clinical condition.

There is no single recommended gold standard to determine the validity of EHR data. 114 Nearly three-quarters (74%) of studies used medical records; more frequently for HF diagnoses (85%) than ACS (71%) or stroke (68%). This difference may be due to availability of MI and stroke registries, used in 26% and 22% of studies, respectively. No differences in the performance of the reference standard methods were discernable, probably due to heterogeneity. Criteria to define CVD, especially MI, have been refined over time, driven by the development of more sensitive and specific biomarkers, and more precise imaging techniques. 100 However, we did not identify any temporal trends in the accuracy of MI recording, again likely due to overall study heterogeneity.

When validating HF, which can vary in clinical aetiology and presentation, clarity on the criteria used to define, with explicit classification of acute and chronic HF along with ejection fraction would benefit understanding of results.

Only 14 (17%) studies validated primary care systems, more than half of which were in the UK. Using primary care EHRs may be beneficial for research into conditions such as HF which are frequently managed in primary care; in our study, 30% of HF EHR validation studies used primary care data, compared to 16% for ACS and 7% for stroke studies. For acute severe conditions resulting in hospitalization, secondary care records should be the most reliable data source. Where possible, the use of linked data to increase the ascertainment of acute CVD events should be considered.

EHR-based research is a growing field -widely used in observational analyses and increasingly employed in trials. 115 Researchers should consider the level of validity necessary for their own CVD outcome definition. When a composite outcome, such as MACE, is used researchers may need to address differing sensitivity in the individual components of the outcome. In studies which investigate CVD incidence, a sensitive definition is particularly important. For example, EHR data are being used for rapid COVID-19 pandemic analyses such as; the impact the virus has in those with CVD, CVD as an outcome after infection with the virus, and excess death estimates. 116 It is important that these rapid analyses consider the validity of the data and definitions used. Conversely, in a pragmatic trial recruitment, a specific definition is likely more important than a sensitive one.

Our systematic review provides a comprehensive and up-todate evaluation of the validity of acute CVD diagnoses in European EHRs, conducted without language or time restrictions using a broad search strategy. Two independent reviewers performed our study selection, and native speaking collaborators translated foreign language articles. Similar to other systematic reviews of validation studies, we repurposed the QUADAS-2 risk of bias tool developed for diagnostic test accuracy. Additionally, we followed the diagnostic test accuracy GRADE methodology to assess the overall evidence base.

Our work is not without limitations. Firstly, only one reviewer completed full data extraction and risk of bias assessment due to resource constraints, although a sample of 20% of studies had data dual extracted. Secondly, we limited our study to Europe, so theoretically our results are only generalizable to European countries. All previous systematic reviews 8-12 on the validity of acute CVD diagnoses included both EHRs and claim-based systems, while most studies included in each of these reviews were from North America. From these existing reviews, it was unclear if the validity of EHRs differed to claims-based datasets, which reflect payments related to medical care given. Despite this, we obtained similar results to the previous reviews. Thirdly, our review focused on acute CVD events so excluded results from studies that validated broader diagnoses of ischaemic heart disease or cerebrovascular disease, which again limits generalizability to these specific conditions.

For ACS and stroke diagnoses, most sensitivity and PPV results were reasonably high, providing confidence in the use of European EHR data for research into these conditions. However, there was considerable heterogeneity between studies. Sensitivity for HF diagnoses was low, and our GRADE assessment found very low quality for all HF outcomes. For studies of HF, we strongly recommend either validating the definition or referring to existing validation studies to develop the case definition. New validation studies of HF diagnoses should report whether the diagnoses validated are for acute or chronic presentation and HF with reduced ejection fraction or preserved ejection fraction. These principles are also applicable to future ACS and stroke validation studies. Identifying specific stroke subtypes can be difficult; analysis of all stroke subtypes combined is preferable.

Our review on the accuracy of HF, ACS and stroke diagnoses in European EHRs should guide researchers in their selection of data sources and CVD definitions for epidemiological studies. Generally, the data assessed was of reasonable quality. However, it is difficult to summarize validity given the heterogeneity between studies. Where possible, researchers should validate data before use or carefully interpret the results of previous validation studies to consider the impact validity has on research findings. Additionally, the use of linked data will bolster quality.

The top 10 causes of death

Big biomedical data and cardiovascular disease research: opportunities and challenges

Validation and validity of diagnoses in the general practice research database: a systematic review

Quality of the Finnish hospital discharge register: a systematic review

External review and validation of the Swedish national inpatient register

The Danish National Patient Registry: a review of content, data quality, and research potential

Use of electronic health records to ascertain, validate and phenotype acute myocardial infarction: a systematic review and recommendations

UK Biobank Follow-Up and Outcomes Working Group. Accuracy of electronic health record data for identifying stroke cases in large-scale epidemiological studies: a systematic review from the UK biobank stroke outcomes group

Validity of myocardial infarction diagnoses in administrative databases: a systematic review

Validity of heart failure diagnoses in administrative databases: a systematic review and meta-analysis

Validity of diagnostic codes for acute stroke in administrative databases: a systematic review

Major adverse cardiovascular events: an inevitable outcome of ST-elevation myocardial infarction? A literature review

Validity of acute cardiovascular outcome diagnoses in European electronic health records: a systematic review protocol

ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure: the task force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC) developed with the special contribution

Case definitions for acute coronary heart disease in epidemiology and clinical research studies

An updated definition of stroke for the 21st century: a statement for healthcare professionals from the

QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies

Cochrane handbook for systematic reviews of interventions. Version 5

Metaprop: a Stata command to perform meta-analysis of binomial data

Grading quality of evidence and strength of recommendations for diagnostic tests and strategies

Accuracy of angina pectoris and acute coronary syndrome in the Danish National Patient Register

Identification of acute myocardial infarction from electronic healthcare records using different disease coding systems: a validation study in three European countries

Accuracy of a heart failure diagnosis in administrative registers

Predictive value of stroke discharge diagnoses in the Danish National Patient Register

The validity of the diagnosis of acute myocardial infarction in routine statistics: a comparison of mortality and hospital discharge data with the Danish MONICA registry

Positive predictive value and impact of misdiagnosis of a heart failure diagnosis in administrative registers among patients admitted to a university hospital cardiac care unit

Positive predictive value of cardiovascular diagnoses in the Danish National Patient Registry: a validation study

The predictive value of ICD-10 diagnostic coding used to assess Charlson comorbidity index conditions in the populationbased Danish National Registry of Patients

Registration of acute stroke: validity in the Danish Stroke Registry and the Danish National Registry of Patients

Validity of Danish register diagnoses of myocardial infarction and stroke against experts in people with screendetected diabetes

The validity of the diagnosis of heart failure (I50.0-I50.9) in the Danish National Patient Register

Evaluation of algorithms for registry-based detection of acute myocardial infarction following percutaneous coronary intervention

Trend in mortality after stroke with atrial fibrillation

Risk of subarachnoid haemorrhage in first degree relatives of patients with subarachnoid haemorrhage: follow up study based on national registries in Denmark

Predictive values of acute coronary syndrome discharge diagnoses differed in the Danish National Patient Registry

Predictive value of stroke and transient ischemic attack discharge diagnoses in The Danish National Registry of Patients

Validity of stroke diagnoses in a National Register of Patients

Correcting outcome data for case mix in stroke medicine

Incidence of stroke and seizure in Alzheimer's disease dementia

A study to determine the sensitivity and specificity of hospital discharge diagnosis data used in the MICA study

Does ICD-10 hospital discharge code I50 identify people with heart failure? A validation study within the EPIC-Norfolk study

Validation of ischemic cerebrovascular diagnoses in the health improvement network (THIN)

Stroke incidence: sensitivity of hospital data coding of acute stroke

The quality of information recorded on a UK database of primary care records: a study of hospitalizations due to hypoglycemia and other conditions

Vascular disease in women: comparison of diagnoses in hospital episode statistics and general practice records in England

Risk of acute myocardial infarction, stroke, or death in patients initiating olmesartan or other angiotensin receptor blockers -a cohort study using the clinical practice research datalink

The accuracy of Scottish Morbidity Record (SMR1) data for identifying hospitalised stroke patients

Optimal strategies for identifying patients with myocardial infarction in general practice

Incidence of hemorrhagic stroke in the general population: validation of data from The Health Improvement Network

Determining the predictive value of Read/OXMIS codes to identify incident acute myocardial infarction in the General Practice Research Database

Completeness and diagnostic validity of recording acute myocardial infarction events in primary care, hospital care, disease registry, and national mortality records: cohort study

Do discharge codes underestimate hospitalisation due to heart failure? Validation study of hospital discharge coding for heart failure

The accuracy of hospital discharge coding for hemorrhagic stroke

Validity of cardiovascular disease event ascertainment using linkage to UK hospital records

Stroke incidence in young adults according to age, subtype, sex, and time trends

A national record linkage to study acute myocardial infarction incidence and case fatality in Sweden

NSAIDs associated with increased risk of congestive heart failure in elderly patients taking diuretics

Occurrence and relative risk of stroke in incident and prevalent contemporary rheumatoid arthritis

Risk of cardiovascular events and death in the life after aneurysmal subarachnoid haemorrhage: a nationwide study

Comparison of the validity of stroke diagnoses in a medical quality register and an administrative health register

Refinement of Swedish Administrative Registers to monitor stroke events on the national level

Validity of coronary heart diseases and heart failure based on hospital discharge and mortality data in the Netherlands using the cardiovascular registry Maastricht cohort study

Measuring stroke in the population: quality of routine statistics in comparison with a populationbased stroke registry

The validity of heart failure diagnoses obtained from administrative registers

The validity of the Finnish Hospital Discharge Register and causes of death register data on coronary heart disease

How to define coronary heart disease in register-based follow-up studies: experience from the Helsinki Heart Study

Validity of diagnoses of major coronary events in national registers of hospital diagnoses and deaths in Finland

Validation of the Swedish inpatient and cause-of-death registers in the context of stroke

Validation of stroke diagnosis in the National Hospital Discharge Register and the Register of Causes of Death in Finland

The validation of the Finnish Hospital Discharge Register and causes of death register data on stroke diagnoses

Administrative data underestimate acute ischemic stroke events and thrombolysis treatments: data from a multicenter validation survey in Italy

Hospital discharge data for assessing myocardial infarction events and trends, and effects of diagnosis validation according to MONICA and AHA criteria

Choice of ICD-10 codes for the identification of acute coronary syndrome in the French hospitalization database

Diagnostic accuracy of the International Classification of Diseases, Tenth Revision, codes of heart failure in an administrative database

Positive predictive value of French hospitalization discharge codes for stroke and transient ischemic attack

Can hospital discharge databases be used to follow ischemic stroke incidence? Stroke

Are geographical differences and time trends in myocardial infarction incidence in Sweden real? Validity of hospital discharge diagnoses

Validity of hospital discharge data in a prospective epidemiological study on stroke and myocardial infarction

Validity of registration of ICD codes and prescriptions in a research database in Swedish primary care: a cross-sectional study in Skaraborg primary care database

The validity of a diagnosis of heart failure in a hospital discharge register

Reliability of hospital discharge diagnoses of acute myocardial infarction

Underestimated prevalence of heart failure in hospital inpatients: a comparison of ICD codes and discharge letter information

Accuracy of the ICD-9 codes for identifying TIA and stroke in an Italian automated database

Validity of register data on acute myocardial infarction and acute stroke: the Skaraborg Hypertension Project

Validation of intracranial hemorrhage in the Norwegian Patient Registry

Abad-Vicente FJ, Merino-Peralta A. Validity of the clinical and administrative databases in detecting post-operative adverse events

Validation of stroke diagnosis in the National Registry of Hospitalized Patients in the Czech Republic

Measuring accuracy of discharge diagnoses for a region-wide surveillance of hospitalized strokes

Overdiagnosis of heart failure in primary care: a cross-sectional study

The effects of misclassification in routine healthcare databases on the accuracy of prognostic prediction models: a case study of the CHA2DS2-VASc score in atrial fibrillation

Validation of heart failure diagnosis registered in primary care records in two primary care centres in Barcelona (Spain) and factors related. A cross-sectional study

Automatic identification of type 2 diabetes, hypertension, ischaemic heart disease, heart failure and their levels of severity from Italian General Practitioners' electronic medical records: a validation study

Clinical and administrative data on the research of acute coronary syndrome in Spain. Minimum basic data set validity

The validity of hospital discharge register data on coronary heart disease in Finland

Diagnosis of acute myocardial infarction by MONICA and FINMONICA diagnostic criteria in comparison with hospital discharge diagnosis

How accurate is the reporting of stroke in hospital discharge data? A pilot validation study using a population-based stroke registry as control

Identification of incident stroke in Norway: hospital discharge data compared with a population-based stroke register

The epidemiology of heart failure: the Framingham Study

An analysis of physicians' reasons for prescribing long-term digitalis therapy in outpatients

ACCF/AHA guideline for the management of heart failure: a report of the American College Of Cardiology Foundation/American Heart Association Task Force On Practice Guidelines

Myocardial infarction and coronary deaths in the World Health Organization MONICA project: registration procedures, event rates, and case-fatality rates in 38 populations from 21 countries in four continents

Universal definition of myocardial infarction. Circulation

Third universal definition of myocardial infarction

Cerebrovascular disease in the community: results of a WHO collaborative study

Stroke trends in the WHO MONICA project

Sensitivity, specificity, positive and negative predictive values of identifying atrial fibrillation using administrative data: a systematic review and meta-analysis

The validity of dementia diagnoses in routinely collected electronic health records in the United Kingdom: a systematic review

How to validate a diagnosis recorded in electronic health records

Big data from electronic health records for early and late translational cardiovascular research: challenges and potential

Estimating excess 1-year mortality from COVID-19 according to underlying conditions and age in England: a rapid analysis using NHS health records in 3.8 million adults

online journal focusing on disease and drug epidemiology, identification of risk factors and screening procedures to develop optimal preventative initiatives and programs. Specific topics include: diagnosis, prognosis, treatment, screening, prevention, risk factor modification, systematic reviews, risk & safety of medical interventions, epidemiology & biostatistical methods, and evaluation of guidelines, translational medicine, health policies & economic evaluations. The manuscript management system is completely online and includes a very quick and fair peer-review system

We thank Hanne-Dorthe Emborg and Elisabeth Bondesson for their translations of Danish and Swedish language articles.

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work. 

Charlotte Warren-Gash reports grants from Wellcome, grants from British Heart Foundation, during the conduct of the study; personal fees from Sanofi Pasteur, outside the submitted work. The authors declare no other potential conflicts of interest for this work.