key: cord-0779812-wd3pty7b authors: Yin, Yuhuan; Gao, Jiangxia; Zhang, Yiyin; Zhang, Xiaoli; Ye, Jianying; Zhang, Juxia title: Evaluation of reporting quality of abstracts of randomized controlled trials regarding patients with COVID-19 using the CONSORT statement for abstracts date: 2022-01-07 journal: Int J Infect Dis DOI: 10.1016/j.ijid.2022.01.002 sha: 420678a32bbe415a7e6479d7ade0f7bda608d3b4 doc_id: 779812 cord_uid: wd3pty7b Objective To evaluate the reporting quality of randomized controlled trial (RCT) abstracts regarding patients with COVID-19 and analyse the influence factors. Methods PubMed, Embase, Web of Science and the Cochrane Library databases were searched to collect RCTs regarding patients with COVID-19. The retrieval time was from the inception to December 1, 2020. The CONSORT statement for abstracts was used to evaluate the reporting quality of RCT abstracts. Results 53 RCT abstracts were included, the CONSORT statement for abstracts showed that the average reporting rate of all items was 50.24%, the lower reporting quality of items were mainly trial design and the details of randomization and blinding (<10%). Mean overall adherence score across all studies was 8.68±2.69 (ranged: 4-13.5). The multivariate linear regression analysis showed the higher reporting scores were associated with higher journal impact factor (P<0.01), international collaboration (P=0.04) and structured format (P<0.01). Conclusions Although many RCTs regarding patients with COVID-19 have been published in different journals, the overall reporting quality of the included RCT abstracts was suboptimal, thus diminishing their potential usefulness, and this may mislead clinical decision-making. In order to improve the reporting quality, it is necessary to promote and actively apply the CONSORT statement for abstracts. With the development of evidence-based medicine, randomized controlled trials (RCTs) became the gold standard to compare the effectiveness of different interventions and provide direct evidence for clinical decision making (Sackett et al., 1996) . It can avoid possible bias in clinical trial design, balance confounding factors and improve the effectiveness of statistical tests (Chalmers, 1998) . Therefore, accurate and complete reporting of RCT results are essential for the effective utilization of high-quality evidence (Cook et al., 1996) . This has led to the development of reporting guidelines for RCTs. The Consolidated Standards of Reporting Trials (CONSORT) was established in 1996 (Begg et al., 1996) , revised in 2001 (Moher et al., 2001) and last updated in 2010 (Schulz et al., 2010) . As the full text of many papers is not available and the limited working time, most readers usually evaluate the results of trials only on the basis of the information in abstracts and decide on the need for further reading of the full text (Schnelle et al., 1992) , and researchers rely heavily on abstracts when deciding whether to include an article in a meta-analysis or systematic review (Smith et al., 2007) . In addition, the results of Marcelo et al's study showed that more than a third of doctors routinely used abstracts to answer clinical questions (Marcelo et al., 2013) . These factors indicate that abstracts are an integral part of the initial assessment of the value of trial, inadequate reporting in abstracts may affect the correct application of study results in everyday clinical practice (Begg et al., 1996; Pitkin et al., 1999; Ioannidis et al., 2001) . Thus, the reporting quality of abstracts are critical. To improve the reporting quality of RCT abstracts, the Consolidated Standards of Reporting for Abstracts (CONSORT-A) was published in 2008 (Hopewell et al., 2008) , which is a reporting guideline extension and provides guidance on the reporting of RCT abstracts. With the publication of the CONSORT for abstract reporting, it was endorsed by the World Editors, and the Council of Science Editors (Hopewell et al., 2008) . The CONSORT statement for abstracts provides the basic information to correctly assess the validity of trials and the applicability of results of RCTs (Hopewell et al., 2008) . COVID-19 emerged in December 2019 and quickly spread to countries around the world. As of 3 September 2021, there have been 218,946,836 confirmed cases of COVID-19, including 4,539,723 deaths (WHO, 2021) . As a new and serious infectious disease threatening human health, its treatment and prevention are of vital importance. Although the CONSORT statement for abstract reporting has been issued, the reporting quality of RCT abstracts has been a subject of concern. Some previous studies have shown that the reporting quality of RCT abstracts was still suboptimal in other fields (Chen et al., 2018; Fang et al., 2020; Hua et al., 2019; Chow et al., 2018) . As poor reporting quality of RCT abstracts was a barrier to evidence-based practices, if RCT abstracts of patients with COVID-19 are also reported poorly, it will affect clinicians' decision making and the effectiveness of evidence-based practices (Begg et al., 1996) , thus detrimental to the treatment of patients and disease control, and may even cause harm. The primary objective of our study was to assess the reporting quality of RCT abstracts regarding patients with COVID-19 and analyze possible related causes, so as to provide reliable evidence for subsequent related studies and meta-analyses, and provide suggestions for RCT abstracts of higher reporting quality. This research was a cross-sectional study that analyzed data from published RCT abstracts regarding patients with COVID-19. PubMed, Embase, Web of Science and the Cochrane Library databases were searched to collect RCTs regarding patients with COVID-19. The retrieval time was from the inception to December 1, 2020. The search was conducted by two investigators and the detailed strategy was shown in S1 Appendix. Studies meeting the following criteria were enrolled in the study: (1) Randomized controlled trials. (2) The confirmed or suspected patients of according to the diagnostic criteria of "the latest clinical guidelines for novel coronavirus" issued by the World Health Organization (WHO). (3) Interventions related to patients or suspected patients. Exclusion criteria included: (1) Animal experiments, reviews, systematic reviews, case reports and conference paper. (2) Repeated publications. (3) The abstract or full text is not available. The titles of the retrieved article were imported into Endnote X9 and screened by 2 reviewers independently. We first reviewed the title and abstract of each article and decided to regard its appropriateness for inclusion. In case of doubt, we downloaded full texts to judge whether an article was indeed an RCT. Any disagreement was solved by consensus. Two authors independently extracted the general characteristics and reporting data of 53 studies into Excel. The general characteristics include continent of first-author, number of authors, sample size, participants, interventions, journal impact factor, international collaboration, word count, structured format. Two authors independently assessed the reporting quality of included abstracts by the original CONSORT-A guideline and relevant explanations (Hopewell et al., 2008) , any discrepancy was resolved through discussion. As we did not include conference abstracts, among the 17 items of the original CONSORT-A checklist, one item (authors) was designed specifically for conference abstracts and therefore excluded from our assessment. Each checklist item was evaluated to analyze whether it was adequately reported, not reported, or unclear in an RCT abstract. We assigned the following scores to each item: "yes (1 point)", "unclear (0.5 point)", or "no (0 point)" to indicate whether it was reported (Hua et al., 2015) . A score of "1" was given if the items 1, 2, 4, 5, 6, 9, 10, 13, 15 and 16 were adequately reported, a score of "0" was given if the item was not reported. Item 3 contains three sub-items a, b and c, items 8, 11 and 14 each contain two sub-items a and b, and items 7 and 12 each contain four sub-items a, b, c and d. A score of "1" was given if the sub-item "a" was reported, a score of "0.5" was given if the sub-items "b, c or d" was reported, a score of "0"was given if these items were not reported, and the sub-items a, b, c, and d are independent of each other. Then for each abstract, an overall CONSORT-A score (score range: 0 to 16) was calculated by totaling the scores of all the sixteen quality items. Statistical analyses were undertaken using the SPSS 21.0 software. Variables were summarized using descriptive statistics namely absolute (n) and relative (%) frequencies for categorical variables and mean ( ̅ ) and standard deviation (SD) for numerical variables. We used the k coefficient to determine the degree of agreement between reviewers. An independent sample T-test and one-way ANOVA were used to compare inter-group differences in CONSORT-A scores for general characteristics, as the data met all the relevant assumptions including normality and homogeneity of variance. Multiple linear regression analysis was performed to determine the association between potential predictors and abstract reporting quality. All significant predictors in the univariable analysis were entered individually into a multivariable analysis. Potential predictors were coded as follows: sample size (＜100=1, ≥ 100=2); interventions (psychology=1, pharmacology= 2); international collaboration (no=1, yes=2); journal impact factor (＜10=1, ≥10=2); international collaboration (no=1, yes=2); word count (≤250=1, >250=2); structured format (no=1, yes=2). The dependent variable was CONSORT-A scores. No significant violation of normality was found in assessments of the residuals. For all analyses, the statistical significance level was set at P<0.05. Ethical approval was not necessary for this study, as the study did not involve patients, and the included RCT abstracts can be traced from databases. Initially, 8700 RCTs were obtained. Excluding duplicates, 6,922 studies was remained. After screening the titles and abstracts, 198 potentially eligible articles were identified. Subsequently full text of each article was retrieved with a total of 53 RCTs that were confirmed for further assessment. Figure 1 outlined the search detail via the PRISMA flow diagram. In the pilot study, inter-observer concordance for article selection had a kappa score of 0.82, which was 0.90 after resolving all disputed items by a discussion with the third reviewer (ZJX), suggesting that inter-observer reliability was almost perfect. Among the included 53 RCTs, most of articles were from Asia and accounted for 62.3%. The number of authors was mainly more than 20. Over 90% of the articles used drug interventions, and 22 studies (42.5%) had the journal impact factor of more than 10. 14 (26.4%) abstracts were limited to 250 words and most abstracts (83.0%) have adopted structural format. As showed in Table 1 . Table 2 described the results of quality assessment using the CONSORT-A guideline. The CONSORT-A checklist showed that the average reporting rate of all items was 50.2%. In the general items, the reporting rates of the title, trial registration and funding were all over 70%, but only 2 (3.8%) studies described trial design in their abstracts. Regarding the methodological items, the average reporting rate was 40.6%. Randomization and blinding had poor reporting quality. Only one (1.9%) study described the methods of generating random sequences, 9 (17.0%) studies mentioned the blinding in their abstracts, but none of the studies have reported the details of blinding. In the results and section, the average reporting rate was 43.4%. More than one-half of abstracts reported the number of participants for randomizing to each group, number of participants analyzed in each group, outcomes and conclusions. But recruitment and harms were poorly reported, only 14 (26.4%) studies described trial status and less than half the studies reported the harms. The scores ranged from 4 (25%) to 13.5 (84%). 53 studies had a mean score of 8.68 [54.3% adherence; 95% CI (5.99-11.36)] and a median score of 8.5. Table 3 showed the results of linear regression modeling. The seven statistically significant predictors in the univariate analysis were entered into a multivariable model. Among these, journals impact factor (P<0.01), international collaboration (P=0.04) and structured format (P<0.01) still persisted as noticeable predictors of the overall CONSORT-A scores. This study evaluated the reporting quality of RCT abstracts regarding patients with COVID-19 and analyse the influence factors, it will provide important baseline information for the quality of RCT abstract reports regarding COVID-19. Our study showed that the overall reporting quality in included RCT abstracts was suboptimal, which is particularly worrying. As abstract is an important part of published articles, it serves as the foundation for the initial screening in any meta-analysis and systematic review, incomplete research reports in abstracts may result in the RCT not being included in systematic reviews (Evans, 2003) . In addition, abstract is also an important tool for clinical decision making, as there may be financial, technical, temporal, and language barriers that impede or reduce access to the full text of the article, in the biomedical field, almost half of all research is available in full text only by subscription (Kurata et al., 2013) , but abstracts are usually free, so many medical professionals base their initial evaluation or even clinical decision-making on abstracts alone (Saint et al., 2000; Ioannidis et al., 2001; Cullen, 2002) . The omission of essential trial details in RCT abstracts can lead to inaccurate interpretation of study results and inappropriate application of results in clinical practice (Sriganesh et al., 2017) . The above factors show that complete, clear and accurate abstract reports based on the CONSORT -A guideline are necessary to help clinicians and wider readers critically appraise RCT outcomes. Although the CONSORT-A was established to ensure the completeness and accuracy of RCT abstract reports, the reporting quality of these RCT abstracts that we included was still poor, with the overall average reporting rate of 50.2%, which was similar to the findings in the fields of Plastic Surgery, Endodontics and Prosthodontics (Alharbi and Almutairi, 2020; Fang et al., 2020; Gallo et al., 2020) . Even RCT abstracts published in top pain journals had an average reporting rate of less than 40% (Sriganesh et al., 2017) , which is quite worrying, and the average reporting rate of methodological section was also poor. The results of this study may be due to the large number of patients with COVID-19 emerging in a short period of time, in order to present positive results of various treatment regimens to readers as soon as possible, researchers may have paid more attention to the results of study than report specifications. Of the 16 items, only two items ( "objective"and "conclusions" ) were adequately reported in most abstracts (>90%), and none of the abstracts provided complete information as required. The reporting quality of most items was suboptimal, particularly including the trial design, randomization and blinding (＜10%). Explanation of the trial design in the abstracts can increase the transparency of study, thereby reducing the possibility of misunderstanding of the data (Calvert et al., 2013) , readers may misinterpret the cohort data as sample size if the trial design was not clearly explained (Campbell et al., 2004) . But our study showed that only two (2.8%) abstracts reported the type of trial design in their abstracts, some previous studies also showed trial design with the reporting rates of 16.3-26.6% (Kuriyama et al., 2017; Janackovic and Puljak, 2018; Khan et al., 2019) . In addition, the neglect of two most important items in the methodological section (randomized methods and details of blinding) is particularly worrisome, as these items are important information to ensure the authenticity of results (Seehra et al., 2013) . Only one study described the details of randomization, 26 (49.1%) abstracts mentioned random assignment but failed to report sequence generation and concealment of allocation, which has not improved since the release of CONSORT-A (Kuriyama et al., 2017; Chen et al., 2018; Hua et al., 2019) . Even RCT abstracts published in high-impact factor journals also had similar flaws. In a reported analysis of 478 RCT abstracts published in the high-impact cardiovascular journals, Khan et al. found that only 3.6% reported the details of randomization (Khan et al., 2019) . What's more, none of the studies reported the details of blinding in our study, we found that some abstracts only indicated that the study was "single" or "double" blind rather than specifying exactly who were unaware of treatment identities, which was similar to the results of prosthodontics and emergency medicine journals (Chen et al., 2018; Germini et al., 2019) . The low reporting rates of these items may reflect the lack of relevant knowledge of researchers to some extent, because the report of sequence generation and concealment of allocation need to have certain knowledge of clinical research methodology (Chen et al., 2010) . There was evidence that trials with inadequate or unclear allocation concealment overestimated the treatment effects up to 7% (Savović et al., 2012) , and a meta-epidemiological study of blinding showed that unblinded RCTs overestimated the outcome effect by 0.56 standard deviations (Hróbjartsson et al., 2014) . Because the lack of report of important methodological items could affect the reliability and validity of RCT abstracts (Berwanger et al., 2009) , these items need to be addressed properly. In addition, according to CONSORT -A, adverse events (harms) are the important piece of information for readers and should be reported in abstracts, if no important adverse events have occurred, the authors should state this explicitly (Hopewell et al., 2008) . Although our study showed that the report of harms was relatively positive compared to the RCT abstracts published in fields of critical care, prosthodontics and endodontics (Kuriyama et al., 2017; Chen et al., 2018; Fang et al., 2020) , it was still less than 50%. As a new and serious infectious disease, the treatment and prevention of COVID-19 are of great importance, if authors failed to report adverse events or side effects in their abstracts, it will mislead the clinical decision of the physicians and may even cause harm to patients (Song et al., 2010) . Reporting harms clearly in abstracts is also important for proper database indexing and information retrieval (Hopewell et al., 2008) . Our study showed that the better reporting quality was associated with international collaboration, similar result have been found in the area of periodontal disease (Xie et al., 2020) . The exact reasons behind this phenomenon were unknown, but we can find that the focus of COVID-19 may come from the whole community and society, not just individuals. However, it can be assumed that researchers from different countries perceive the importance of abrtract reports differently, higher recognition and dependence of CONSORT-A can improve the quality of abstract reports. Our found that better reporting quality was also associated with the structured format, which are consistent with study results in the fields of prosthodontics, dentistry and psychiatry (Chen et al., 2018; Sharma and Harrison, 2006; Song et al., 2017) . Structural abstracts can enhance the reader's understanding for the article (Nakayama et al., 2005) , assist health professionals find clinically relevant articles more quickly and conduct a more detailed literature search (Fontelo et al., 2013) , and also speed up the peer-review process before publication. Although originally intended to facilitate computerized search, structured abstracts have proven to provide more information of the study than unstructured ones (Mbuagbaw et al., 2014; Kiriakou et al., 2014) . Unfortunately, we found that 9 (17.0%) studies did not use a structured abstract, which may be affected by journal submission requirements. Our study showed that journal impact factor was also an important factor for the reporting quality of RCT abstracts, better reporting quality was associated with higher journal impact factor, the results of several previous studies also supported our findings (Chen et al., 2018; Bigna et al., 2016) . This may be due to the fact that journals with higher impact factor have more stringent control about the acceptance and publication of papers, studies showed that higher impact factor was independently associated with better adherence to the CONSORT-A guideline (Ghimire et al., 2014; Lee et al., 2013) . However, we found that most journals used the CONSORT statement as a tool to assess the reporting quality for full-text of RCTs, but failed to endorse the CONSORT-A guideline. Therefore, some authors were not initially aware of the CONSORT-A, they may prefer to concentrate on the manuscript, or following the abstract guideline may be considered an extra "job" for the authors (Cobo et al., 2011) . Because abstract plays an important role in clinical decision making, we suggest that first of all, editors should carefully assess whether their journals' submission requirements are normative, journals should not only endorse the CONSORT guideline, and endorsement of the CONSORT-A guideline is also essential. More importantly, journal should require authors to upload the CONSORT-A checklist as a key material for the initial screening when submitting their RCTs. Subsequently, there is a need for more aggressive enforcement of CONSORT-A for journals by strengthening or altering the peer-review process, such as, peer reviewers should check the completeness and accuracy of CONSORT-A checklist when reviewing RCTs, editorial boards should also increase their oversight of the entire process from submission to publication, articles of lower reporting quality will not be published. However, in addition to problems in the whole process of submission requirements, review and publication, another potential reason for the poor reporting quality of RCT abstracts may be that journals do not publicize CONSORT-A guideline enough, resulting in the lack of awareness of reporting guideline by researchers (Reveiz et al., 2013) . A survey of the authors of 101 studies showed that only 3% authors acknowledged the importance of RCT abstract reports and followed CONSORT-A guideline when writing papers (Reveiz et al., 2013) , this suggests that improving researchers' awareness by accelerating the spread of CONSORT-A is crucial. Therefore, on the one hand, journals should vigorously promote the CONSORT-A guideline and can add relevant knowledge of abstract reports to their subscription feeds. On the other hand, research institutions should also increase training in these problems to improve the reporting quality of RCT abstracts, thereby providing scientific evidence for clinical decision-making and meta-analysis. Our study has some limitations. First of all, this study only included relevant articles from 4 databases, it could not represent the overall reporting quality of RCT abstracts regarding patients with COVID-19, but our results may sufficiently reflect the overall trends in the abstract reports of RCTs regarding COVID-19. Second, our study analyzed the adequacy of reports based on the CONSORT for abstracts checklist, without considering whether the content of abstract was accurately reflected in the full text, this was beyond the scope of our study. Thus, further studies are needed to assess the accuracy of the full-text reports. Finally, we only evaluated the RCT abstracts for the treatment of patients with COVID-19, future studies can further assess the reporting quality of more RCTs related to COVID-19. The primary objective of our study was to provide readers a broad overview of the reporting characteristics of RCT abstracts regarding patients with COVID-19. The overall reporting quality of RCT abstracts was suboptimal, thus diminishing their potential usefulness, and it can not provide scientific evidence for clinical decision-making and systematic reviews. Better reporting quality was associated with higher journal impact factor, structured format and international collaboration. More journals should endorse the CONSORT statement for abstracts and strictly monitor the publication of RCTs. Future RCT abstracts should particularly focus on improvement of trial design, randomization and blinding. No authors have any conflict of interest. The conception and design of the study: Yuhuan Yin, Jiangxia Gao, Juxia Zhang. Are published randomized clinical trials abstracts on periodontics reported adequately? Improving the quality of reporting of randomized controlled trials. The CONSORT statement The quality of reporting of trial abstracts is suboptimal: survey of major general medical journals Abstracts reporting of HIV/AIDS randomized controlled trials in general medicine and infectious diseases journals: completeness to date and improvement in the quality since CONSORT extension for abstracts Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension CONSORT statement: extension to cluster randomised trials Unbiased, relevant, and reliable assessments in health care: important progress during the past century, but plenty of scope for doing better Quality improvement in randomized controlled trial abstracts in prosthodontics since the publication of CONSORT guideline for abstracts: a systematic review Assessment of the quality of reporting in abstracts of randomized controlled trials published in five leading Chinese medical journals The degree of adherence to CONSORT reporting guidelines for the abstracts of randomised clinical trials published in anaesthesia journals: A cross-sectional study of reporting adherence in 2010 and 2016 Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial Substance abuse prevention in the workplace: Recent findings and an expanded conceptual model In search of evidence: family practitioners' use of the Internet for clinical information Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions Abstracts of published randomised controlled trials in Endodontics: Reporting quality and spin Comparing data accuracy between structured abstracts and full-text journal articles: implications in their use for informing clinical decisions The Reporting Quality of Randomized Controlled Trial Abstracts in Plastic Surgery Quality of reporting in abstracts of RCTs published in emergency medicine journals: a systematic survey of the literature suggests we can do better Oncology trial abstracts showed suboptimal improvement in reporting: a comparative before-and-after evaluation using CONSORT for Abstract guidelines CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies Reporting quality of randomized controlled trial abstracts: survey of leading general dental journals Reporting quality of randomised controlled trial abstracts presented at the SLEEP Annual Meetings: a cross-sectional study Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas Reporting quality of randomized controlled trial abstracts in the seven highest-ranking anesthesiology journals Assessing the Quality of Abstracts in Randomized Controlled Trials Published in High Impact Cardiovascular Journals Assessing the reporting quality in abstracts of randomized controlled trials in leading journals of oral implantology Remarkable growth of open access in the biomedical field: analysis of PubMed articles from Reporting of critical care trial abstracts: a comparison before and after the announcement of CONSORT guideline for abstracts Compliance of randomized controlled trials in trauma surgery with the CONSORT statement A comparison of the accuracy of clinical decisions based on full-text articles and on journal abstracts alone: a study among residents in a tertiary care hospital Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: a systematic review The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials Adoption of structured abstracts by general medical journals and format for a structured abstract Accuracy of data in abstracts of published research articles Compliance with clinical trial registration and reporting guidelines by Latin American and Caribbean journals Evidence based medicine: what it is and what it isn't Journal reading habits of internists Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies Reducing and managing restraints in long-term-care facilities CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials Reporting quality of abstracts of randomized controlled trials published in dental specialty journals Structured abstracts: do they improve the quality of information in abstracts? Access to electronic health knowledge in five countries in Africa: a descriptive study Dissemination and publication of research findings: an updated review of related biases Assessing reporting quality of randomized controlled trial abstracts in psychiatry: Adherence to CONSORT for abstracts: A systematic review Quality of abstracts of randomized control trials in five top pain journals: A systematic survey COVID-19) Situation dashboard Quality assessment of randomized controlled trial abstracts on drug therapy of periodontal disease from the abstracts published in dental Science Citation Indexed journals in the last ten years We thank Dr. Bin Ma (Evidence-Based Medicine Center, Institute of Traditional Chinese and Western Medicine, School of Basic Medical Sciences, Lanzhou University, Lanzhou, Gansu, China) for providing assistance with editing the final manuscript.