key: cord-0801214-jorpgamq
authors: Tovey, David; Tugwell, Peter
title: On outcomes
date: 2021-09-03
journal: J Clin Epidemiol
DOI: 10.1016/j.jclinepi.2021.08.015
sha: 7ab368bcd1c5242a16d882005d6d003fa96d4ffb
doc_id: 801214
cord_uid: jorpgamq

nan

Outcomes are the meeting point between evidence and clinical practice and health policy decisions. They need to be valid, to represent those that are the most important to decision makers (including patients and the public), and they need to be reported accurately, without spin or inappropriate selection. In the August issue of JCE we include four papers that address two important outcome related issues: the identification and reporting of harmful effects of interventions, and the use of core outcome sets to improve the evidence base. Both are of particular relevance in addressing the current Coronavirus pandemic, but they also represent issues of long standing importance to the adoption of evidence informed health care.

Core outcomes sets have become well established over the past decade. They aim to represent the minimum set of outcomes that should be measured and reported in primary studies and systematic reviews in a given area. For this reason, in the context of a novel disease, the early development of core outcomes sets was invaluable to those researchers and decision makers seeking to make sense of the hastily developed research to combat COVID-19.

[REF] The Core Outcome Measures for Effectiveness Trials (COMET) database is a freely available, frequently updated online resource that can easily be used to identify published core outcome sets. In their original article published in the August issue of JCE, Saldana and colleagues attempted to match the outcomes recommended in core outcome sets with those presented in relevant systematic reviews.

[10496] They also developed a relevance framework based on the degree of matching of populations and interventions between paired core outcome sets and systematic reviews, which sought to determine the likelihood of relevance of the former to the latter. They found that 'more than half (54%) of the recent SRs published by the AHRQ Evidence-based Practice Center Program were on topics with relevant core outcome sets'. In the set of reviews examined, slightly fewer than half of the outcomes presented were either an exact or near match to the ones included in the matching core outcomes set. Notably, whilst health care professionals were the group most represented in developing the core outcome sets, more than half included either researchers (57%) or patients (51%), whilst policy makers and guidelines developers were much less frequently involved. The figure relating to patients is particularly noteworthy given the increasing recognition of their crucial role in determining the outcomes that matter most. [REF] The relevance framework centres on a determination of whether the participants and interventions within the selected Core Outcome Set (COS) was an exact match, broader or narrower than the SR, or different but related. Combining the various options led to 16 distinct scenarios that were grouped into those COS that were 'very likely', 'likely' or 'unlikely' to be relevant to the chosen SR. The authors note, surely appropriately, that clinical considerations are crucial in making the assessments of scope matching, and specifically request feedback on the framework and call for its evaluation in practice. Given the likely increasing degree of matching between COS and SRs, it is tempting to ask whether the current guidance to primary researchers and reviewers should be strengthened to recommend actively seek out core outcome sets when they are developing their research protocol, to report the findings of such a search and to explain the rationale for any decision not to include any or all selected outcomes. The Cochrane Methodological Expectations of Cochrane Intervention Reviews (MECIR) conduct standards promote such use, [ref MECIR] but other reporting guidelines appear to fall short of this currently. There may, as the study authors acknowledge be legitimate concerns about adequacy of methodological approach, currency, or inclusiveness of involvement in the development process in individual cases, but it seems a reasonable expectation that trialists or systematic reviewers should clarify this.

In a further paper, a separate but overlapping research group report on a detailed investigation into the inclusion of patient reported outcomes (PROs) in core outcome sets. [10444] . As the study authors note, the assessment of PROs is increasingly seen as an enabler of patient centred healthcare. Despite aims to harmonise and align outcomes, they 'found a fragmented landscape' of patient reported outcome measures (PROM), the vast majority of which were recommended only in one COS, and in a quarter of cases required a single patient to respond to over 100 questions in order to complete the proposed questionnaire. There is evidently scope for harmonisation or standardisation, and the Patient-Reported Outcomes Measurement Information System (PROMIS) developed by the US National Institutes of Health is presented as an example of an approach that seeks to be applicable across disease ar-eas. Disappointingly, these were rarely used in the sample of reviews examined. The authors also reported concerns about validation of PROMs, particularly those based on single question responses, for some of which it was not even possible to locate the exact wording of the chosen question, which is an obvious barrier to use. The article concludes with a call to action by COS developers and methodologists to address the issues identified.

Concerns about the identification and reporting of harms precede even the development of core outcome sets. The commentary by Vohra and a stellar team of researchers confirm that there remains a long way to go before we can be confident that harms are captured and reported as transparently and completely as are beneficial outcomes, in both primary research or evidence syntheses. [10497] . The study explores the extent to which the CONSORT Harms reporting extension has influenced subsequent trials. The results show that there has only been a 'slight' improvement since the extension was published. They note the continuing frequency with which the misleading term 'safety' is used in trial reports, albeit along with the term 'side effects' it has decreased since the extension was published. All of this is in contrast to reports that endorsement of the main CONSORT statement in journals' submission requirements has led to an important improvement in the adherence to good practice amongst journal submissions. The authors recommend that journal editors promote the CONSORT harms extension in the author guidance, and also that the extension should be integrated into the main CONSORT statement. In the context of a pandemic, it is hard to disagree with their conclusion that 'it has passed the time for trialists and the scientific community to recognise the relevance of harms for patients and healthcare decisions.'

Continuing this trend, Papaioannou and colleagues report on their review of adverse event inclusion in trial protocols of behavioral, lifestyle and psychological therapy interventions.

[10443] They note that there are no specific standards or guidelines for recording adverse events in non-pharmacological trials, and that assessments are frequently but inappropriately modelled from the guidance aimed at pharmacological studies. Out of 151 protocols examined, in over a quarter it was not clear whether non serious (27%) or serious adverse events (36%) would be reported. It may be tempting to respond by asserting that non-pharmacological interventions are unlikely to be associated with important harms, but the review shows that this is not true. Adverse events such as suicide, violence, self harm or stigmatisation of young vulnerable individuals are all noted as potential harms in the examples shown. The authors conclude by presenting 5 key recommendations to address the wide variability and lack of transparency in the monitoring and recording of adverse events in nonpharmacological intervention studies.

WHO Working Group on the Clinical Characterisation and Management of COVID-19 infection: A minimal common outcome measure set for COVID-19 clinical research

Core outcome set for clinical trials on coronavirus disease 2019 (COS-COVID). Engineering

Core outcome set for clinical trials of COVID-19 based on traditional Chinese and Western medicine

Core Outcomes Set for Trials in People With Coronavirus Disease

More than half of systematic reviews have relevant core outcome sets

OMERACT: An international initiative to improve outcome measurement in rheumatology

Methodological Expectations of Cochrane Intervention Reviews

Patient-reported outcome measures in core outcome sets targeted overlapping domains but through different instruments

Commentary: Time to improve the reporting of harms in randomized controlled trials

Adverse event recording failed to reflect potential harms: a review of trial protocols of behavioral, lifestyle and psychological therapy interventions