key: cord-0002164-6gi3o5mu
authors: Faber, Timor; Ravaud, Philippe; Riveros, Carolina; Perrodeau, Elodie; Dechartres, Agnes
title: Meta-analyses including non-randomized studies of therapeutic interventions: a methodological review
date: 2016-03-22
journal: BMC Med Res Methodol
DOI: 10.1186/s12874-016-0136-0
sha: a9f3b2f615dda7429498a6f82b0a76ecefa5dcdb
doc_id: 2164
cord_uid: 6gi3o5mu

BACKGROUND: There is an increasing number of meta-analyses including data from non-randomized studies for therapeutic evaluation. We aimed to systematically assess the methods used in meta-analyses including non-randomized studies evaluating therapeutic interventions. METHODS: For this methodological review, we searched MEDLINE via PubMed, from January 1, 2013 to December 31, 2013 for meta-analyses including at least one non-randomized study evaluating therapeutic interventions. Etiological assessments and meta-analyses with no comparison group were excluded. Two reviewers independently assessed the general characteristics and key methodological components of the systematic review process and meta-analysis methods. RESULTS: One hundred eighty eight meta-analyses were selected: 119 included both randomized controlled trials (RCTs) and non-randomized studies of interventions (NRSI) and 69 only NRSI. Half of the meta-analyses (n = 92, 49 %) evaluated non-pharmacological interventions. “Grey literature” was searched for 72 meta-analyses (38 %). An assessment of methodological quality or risk of bias was reported in 135 meta-analyses (72 %) but this assessment considered the risk of confounding bias in only 33 meta-analyses (18 %). In 130 meta-analyses (69 %), the design of each NRSI was not clearly specified. In 131 (70 %), whether crude or adjusted estimates of treatment effect for NRSI were combined was unclear or not reported. Heterogeneity across studies was assessed in 182 meta-analyses (97 %) and further explored in 157 (84 %). Reporting bias was assessed in 127 (68 %). CONCLUSIONS: Some key methodological components of the systematic review process—search for grey literature, description of the type of NRSI included, assessment of risk of confounding bias and reporting of whether crude or adjusted estimates were combined—are not adequately carried out or reported in meta-analyses including NRSI.

Randomized controlled trials (RCTs) are considered the gold standard for evidence-based medicine because they are designed to minimize the risk of bias [1] . However, the applicability of their results has been criticized because of restrictive selection criteria, with, commonly, exclusion of older adults and people with co-morbidities or severe disease [2] [3] [4] . Also, conducting an RCT is sometimes impossible or inappropriate (eg, when studying rare or long-term events) [1, 3, 5] , which results in critical information gaps.

In contrast, observational studies, the overarching term for all non-experimental non-randomized studies (including cohort, case-control, and cross-sectional studies) [6], generally are more likely to reflect clinical practice in real life because of their broader range of participants, longer follow-up time, and lower costs than RCTs [7] [8] [9] [10] . With the aim of generating evidence that will guide healthcare decisions, the field of comparative effectiveness research (CER) emphasizes the need to incorporate data from observational studies to complement RCTs [8, [11] [12] [13] [14] [15] [16] . A comprehensive assessment in 2009 indicated that 54 % of CER studies had an observational study design [17] . Therefore, an increasing number of systematic reviews and meta-analyses are including data from non-randomized studies to assess therapeutic interventions.

Similar to systematic reviews of RCTs, reviews including non-randomized studies are expected to follow the general recommendations for good conduct, such as retrieving all relevant studies and assessing their risk of bias. However, some elements should be adapted specifically to the inclusion of non-randomized studies because their study designs inherently differ from RCTs [7, 9, 14, [18] [19] [20] [21] [22] [23] . Lacking randomization, they are likely subject to confounding bias, which results in an imbalance in prognostic factors associated with the outcome of interest that may severely compromise the validity of their results [24] .

Previous methodological reviews evaluating systematic reviews including observational studies exist [25] [26] [27] [28] . However, these studies have a different objective. One assessed the main characteristics of all systematic reviews indexed in Medline on November 2004 whatever the type of included studies (ie, therapeutic, epidemiological, prognostic or diagnostic studies) [27] . Two others focused on the methods and reporting of harms in systematic reviews of adverse events [26, 28] . The last one was in the field of psychiatry and did not concern therapeutic evaluation but assessment of prevalence or association [25] . Further, none of these previous reviews has evaluated the specific methodological problems raised by the inclusion of non-randomized studies.

In this study, we performed a methodological review of meta-analyses including non-randomized studies of interventions (NRSI) to evaluate key methodological components common to all meta-analyses and those specifically related to the inclusion of non-randomized studies.

This is a methodological review of meta-analyses including NRSI for therapeutic evaluation. For clarity and consistency, we refer to this article as a "methodological review", the systematic reviews with meta-analyses included in this methodological review as "meta-analyses", and the studies included in these meta-analyses as "studies".

Our goal was not to create an exhaustive list of all metaanalyses that include NRSI but rather to identify a relatively representative sample of recently published meta-analyses that a health professional would most likely encounter when searching for meta-analyses. We therefore searched MED-LINE via PubMed because of its wide use among health professionals, combining keywords and MeSH terms for NRSI, systematic reviews, and meta-analyses (Appendix 1). The search was conducted on January 7, 2014 and restricted to the year 2013.

To be eligible, a meta-analysis had to examine a therapeutic or preventive intervention (such as vaccines) for efficacy or safety, include data from at least one NRSI, and be published in 2013. We excluded meta-analyses that included studies without a comparison group and meta-analyses of etiological assessment. When it was difficult to distinguish an etiological from a therapeutic evaluation, we agreed to include the former if the authors considered the inclusion of RCTs in their meta-analysis. To illustrate: a metaanalysis that investigated the association of the use of statins and risk of cancer would was considered a therapeutic evaluation if the authors planned to include RCTs. Individual patient data meta-analyses were also excluded, as were non-randomized studies that conducted a meta-analysis of the literature as secondary analysis. Finally, we did not include meta-analyses published in a language other than English or those for which the full text was not available.

The selection of relevant meta-analyses was conducted in 2 steps. In the first step, one reviewer (CR) excluded clearly irrelevant studies based on the title, abstract, and full text, then, a second reviewer (TF) performed the final selection, discussing all doubtful cases with a third reviewer (AD).

The data extraction form for this methodological review was developed from the MOOSE statement for reporting meta-analyses that include observational studies [29] , the PRISMA statement for reporting systematic reviews and meta-analyses of studies evaluating healthcare interventions [30, 31] , and the AMSTAR measurement tool for assessing the methodological quality of systematic reviews [32] . The data extraction form was tested by one reviewer (TF) with 10 studies before data extraction commenced.

Two reviewers (TF, CR) independently extracted all data in duplicate, resolving discrepancies with a third reviewer (AD) if necessary. The following characteristics were extracted from the full text and online appendix of each meta-analysis:

General characteristics: We collected whether the journal was a specialty or a general journal, the location of the corresponding author, and the medical area. We verified whether the metaanalysis was registered on the international prospective register of systematic reviews by the University of York's Centre for Reviews and Dissemination (PROSPERO). We collected whether epidemiologists or statisticians were involved, relying on the definition given by Delgado-Rodriguez et al. [33] and assessed whether the authors reported the funding sources and declared conflict of interests. We assessed whether the meta-analyses evaluated a pharmacological or nonpharmacological intervention. Non-pharmacological interventions were classified as surgical procedures or other interventions. We also assessed the type of studies included: only NRSI or both NRSI and RCTs.

▪ Search strategy: We collected how many and which electronic databases were searched, and whether the search strategy for at least one database was provided. We collected whether reference lists and journals were hand-searched and whether the authors searched for grey literature, and if yes, how: search of registries (eg, ClinicalTrials.gov), conference abstracts, or contacting experts. We assessed whether the authors restrict their searches by language. ▪ Study selection and data extraction process: We assessed whether study selection and data extraction were conducted in duplicate. ▪ Contact of the study authors: We noted whether it was mentioned that study authors were contacted for clarification or additional results. ▪ Methodological quality/risk of bias assessment:

We assessed whether methodological quality or risk of bias assessment was conducted, what tools were used, and whether the assessment was conducted in duplicate.

Meta-analysis methods: ▪ Studies combined: We assessed the types of NRSI included. NRSI were categorized as concurrent (prospective) cohort, nonconcurrent (retrospective) cohort, case-control, or historically controlled studies according to the definition by Ioannidis et al. [21] . We also assessed whether the authors combined the results from NRSI and RCTs and whether they combined results from different types of NRSI (eg, cohort and case-control studies). ▪ Meta-analysis model: We collected whether the authors used crude or adjusted estimates for NRSI and whether they used fixed-or random-effects models to pool the data. For adjusted estimates, we also collected whether the confounding factors taken into account were listed. ▪ Assessment and exploration of heterogeneity:

We collected whether and how the authors assessed heterogeneity and whether they conducted meta-regression, subgroup, or sensitivity analyses to explore heterogeneity.

▪ Assessment of reporting bias: We collected information on whether the authors assessed reporting bias, and how.

The analysis of the data consisted of descriptive statistics, providing numbers and percentages for qualitative variables and median (minimum, maximum, or interquartile range) for quantitative variables. The results were stratified for meta-analyses including only NRSI and those including both NRSI and RCTs. We did not assess statistical differences between these strata. 

Our MEDLINE search identified 3602 citations; Among the 341 potentially relevant meta-analyses, 188 were eligible for this review (Fig. 1 ). Complete references for the included meta-analyses and meta-analyses excluded are in Appendixes 2 and 3, respectively.

General characteristics (Table 1) Among the 188 included meta-analyses, 49 (26 %) were of surgery, 33 (18 %) cardiology, and 25 (13 %) oncology. Half of the meta-analyses assessed non-pharmacological interventions (n = 92, 49 %); 74 involved surgical procedures. Approximately one third (n = 69, 37 %) included only NRSI, and two thirds included both NRSI and RCTs (n = 119, 63 %).

In total, 36 meta-analyses (19 %) involved epidemiologists or statisticians. Conflict of interest was declared in 166 (88 %), with 26 reporting a potential conflict of interest. About one-third of the meta-analyses did not report a source of funding (n = 69, 37 %).

Overall, all but one of the meta-analyses reported the search of at least 1 electronic database and 147 (78 %) reported the search of > 2 electronic databases. One third provided the search strategy for each database (n = 62, 33 %). MEDLINE, Embase, and the Cochrane Library were most frequently searched (187 [99 %], 149 [79 %], and 126 (67 %) meta-analyses, respectively). In addition to the search of electronic databases, 162 meta-analyses (86 %) reported screening the reference lists of included studies, and 12 (6 %) reported hand-searching journals. About one-third of the meta-analyses (n = 72, 38 %) reported searching for grey literature: 41 (22 %) conference abstracts, 33 (18 %) registries, and 15 (8 %) contacted experts. For 82 meta-analyses (44 %), the authors reported that they did not restrict their searches by language.

Methodological quality or risk of bias of included studies was assessed in 135 (72 %) meta-analyses.

For the 119 meta-analyses including RCTs and NRSI, risk of bias was assessed in 88 (74 %) with 4 assessing risk of bias for RCTs only. RCTs were assessed with the Cochrane Risk of Bias tool in 42 (35 %) meta-analyses. The assessment of risk of bias involved the same tool for both RCTs and NRSI in 27 (23 %) meta-analyses. For the assessment of NRSI, a variety of tools were used. The most frequently used tool was the Newcastle-Ottawa Scale (n = 68). GRADE and the Cochrane Collaboration Risk of Bias Tool were used in 13 and 10 meta-analyses, respectively. In 37 meta-analyses, authors used other tools; in 12, they developed their own tools; and in 12, they were unclear about the methods used for assessing methodological quality/risk of bias. Overall, the authors have considered the risk of confounding bias in their risk of bias assessment in 33 meta-analyses (18 %). Of the 135 meta-analyses with an assessment of risk of bias, 87 (64 %) reported having performed it in duplicate. 

For 130 meta-analyses (69 %), the authors did not clearly report the design for each individual study. Among the metaanalyses that included both NRSI and RCTs (n = 119), for 88 (74 %), the results of NRSI and RCTs were combined.

Concerning NRSI combined, 52 meta-analyses (28 %) included only cohort studies and 5 only prospective cohort studies; 46 meta-analyses (24 %) combined cohort and case-control studies, and 23 (12 %) included all types of NRSI. The other 67 meta-analyses (36 %) included "observational studies" (without further details) (n = 28, 15 %), "prospective and retrospective studies" (n = 23, 12 %), and only "retrospective studies" (n = 16, 9 %).

For 131 meta-analyses (70 %), whether crude or adjusted estimates of treatment effect from the NRSI were used for the meta-analysis was unclear or not reported. For the remaining meta-analyses, the authors reported combining crude and adjusted estimates for 22 (12 %) , only adjusted estimates for 21 (11 %) , and only crude estimates for 6 (3 %). For 8 meta-analyses (4 %), the authors 

NRSI non-randomized studies of intervention; RCTs randomized controlled trials extracted both the crude and adjusted estimates and used them separately in 2 meta-analyses. Among the 51 meta-analyses involving adjusted estimates, 17 (33 %) did not report the confounding factors adjusted for.

A random-effects model was used for half of the metaanalyses (n = 95). For 52 (28 %), a fixed-effects model was used primarily but then replaced with a randomeffects model if high heterogeneity was observed in the model. For 26 meta-analyses (14 %), the authors used both fixed-and random-effects models, and for 9 (5 %), a fixed-effects model. The type of model was not reported or was unclear for 6 meta-analyses (3 %). We found 2 network meta-analyses (1 %).

Almost all meta-analyses assessed heterogeneity (n = 182, 97 %). The I 2 statistic was used in 164 meta-analyses (87 %), Cochran Q χ 2 test in 115 (61 %), and between-study variance τ 2 in 6 (3 %). Heterogeneity was explored in 157 meta-analyses (84 %) by subgroup analyses (n = 126, 67 %), sensitivity analyses (n = 109, 58 %) and meta-regression analyses (n = 34, 18 %). For 44 of 88 (50 %) meta-analyses combining results from RCTs and NRSI, a subgroup or sensitivity analysis was based on the type of study (RCT vs NRSI). For 28 meta-analyses (15 %), subgroup or sensitivity analyses were based on the type of NRSI included.

Reporting bias was assessed in 127 meta-analyses (68 %) by standard funnel plots (n = 111, 59 %), Egger's test (n = 68, 36 %), or Begg's test (n = 42, 22 %). Overall, 82 of the 105 meta-analyses (78 %) including 10 or more studies reported having assessed reporting bias.

We systematically assessed key methodological components of a large sample of therapeutic metaanalyses including NRSI in a variety of medical areas. Our results highlight some important methodological shortcomings. Only 38 % of the meta-analyses reported having searched for grey literature. Specific points related to the inclusion of NRSI raise concerns, with 69 % of the meta-analyses not reporting the study design of the included NRSI, and 70 % not reporting whether crude or adjusted estimates were combined.

To the best of our knowledge, no previous study has comprehensively assessed both key methodological components common to all systematic reviews and elements specific to the inclusion of non-randomized studies.

Other studies that previously evaluated methods or reporting of systematic reviews including NRSI concentrated on the reporting of harms [26, 28] and on systematic reviews in psychiatric epidemiology [25] .

Our study has some limitations. The representativeness of our sample could be debated because we searched for studies in only one online database (MEDLINE), and limited our selection to meta-analyses in English. In addition, for the assessment of the methods, we depended completely on the reporting; we did not assess protocols or contact the authors if methods were not clearly reported. Even though poor reporting does not necessarily reflect poor conduct, it may severely limit the reader's comprehension of the systematic review process [34] .

Before being able to apply the results of any metaanalysis to patient care, health professionals need to evaluate the credibility of the methods of the metaanalysis [35] . One of the key methodological elements is the search for relevant studies. Because not all studies (and particularly those with negative results) are published in scientific journals, a meta-analysis must involve a search for grey literature to try to avoid such publication bias (a type of reporting bias) [24, 35] . However, we found that only 38 % of our meta-analyses reported having searched for grey literature. Because of no mandatory registration for NRSI as for RCTs, most NRSI are not registered, so searching for grey literature of NRSI is difficult [36] . However, a recent study found that for 32 % of the observational studies registered at ClinicalTrials.gov, unpublished results could be retrieved [37] . In contrast, we found that many meta-analyses assessed reporting bias (68 %). Reviewers may have compensated for the absence of searching for grey Only cohort studies 18 (26) 34 (29) 52 (28) Including also case-control studies 18 (26) 28 (23) 46 (24) Including all types of NRSI 5 (7) 18 (15) 23 (12) Other 28 (41) 39 (33) 67 (36) "Observational studies" 6 (9) 22 (18) 28 (15) "Prospective and retrospective studies" 11 (16) 12 (10) 23 (12) "Retrospective studies" 11 (16) 5 (4) 16 (9) Did not clearly report design for each study 43 literature by assessing reporting bias. Evaluating reporting bias does not exempt the reviewers from searching for grey literature because the assessment of Funnel plot asymmetry may be subjective and statistical methods to test for asymmetry of the plot may lack power [38, 39] . Another critical part of the systematic review process is assessing the methodological quality or risk of bias of the studies included, because the validity of the meta-analysis could be questionable with problems in the design and conduct of individual studies [40] . We found that 72 % of our meta-analyses reported having assessed the methodological quality or risk of bias but only 33 (18 %) considered the risk of confounding bias in their assessment. The Cochrane Collaboration has recognized the need to improve the assessment of risk of bias for NRSI and is currently developing a tool for this.

Finally, we found specific issues related to the inclusion of NRSI. In 69 % of the meta-analyses, the study design for each included study was unclear. The risk of bias may vary depending on the type of NRSI, with case-control studies generally considered as having a higher risk of bias than cohort studies. A description of the type of studies included in the meta-analysis is crucial. In addition, NRSI are prone to confounding: an imbalance in prognostic factors associated with the outcome of interest [24] . NRSI are expected to at least present adjusted estimates from multivariate analyses [3, 4] . Many of our meta-analyses (70 %) did not report or were unclear about whether the crude or adjusted estimates of NRSI were combined. Among the meta-analyses involving adjusted estimates, 33 % did not report the confounding factors adjusted for. This information was likely poorly reported in the individual studies, but then the reviewers should contact the authors for clarification or report it clearly in the meta-analysis.

Some key methodological components of the systematic review processsearch for grey literature, description of the type of NRSI included, assessment of risk of confounding bias and reporting of whether crude or adjusted estimates were combined-are not adequately reported in meta-analyses including NRSI. Attention should be paid to improving these elements in such meta-analyses to have an increased confidence in their results.

Not applicable. This article reports a meta-research study.

Not needed. This study does not include human participants.

Data are available upon request for academic researchers. 

Study ID

The preventive effect of statin therapy on new-onset and recurrent atrial fibrillation in patients not undergoing invasive cardiac interventions: a systematic review and meta-analysis

An institutional analysis and systematic review with meta-analysis of pneumatic versus hydrostatic reduction for pediatric intussusception

Bayesian methods affirm the use of percutaneous coronary intervention to improve survival in patients with unprotected left main coronary artery disease

Same-day discharge after percutaneous coronary intervention: a meta-analysis

Efficacy and effectiveness of seasonal and pandemic A (H1N1) 2009 influenza vaccines in low and middle income countries: a systematic review and meta-analysis

Systematic review and meta-analysis of transcatheter aortic valve implantation versus surgical aortic valve replacement for severe aortic stenosis

Drug-eluting stents versus coronary artery bypass graft surgery in left main coronary artery disease: a meta-analysis of early outcomes from randomized and nonrandomized studies

Optimal dosing of antibiotics in critically ill patients using continuous/extended infusions: a systematic review and meta-analysis

Association of blood transfusion with increased mortality in myocardial infarction: a meta-analysis and diversity-adjusted study sequential analysis

Transanal endorectal pull-through versus transabdominal approach for Hirschsprung's disease: a systematic review and meta-analysis

Use of peritoneal dialysis in AKI: a systematic review

A systematic review and meta-analysis of biological treatments targeting tumour necrosis factor alpha for sciatica

Stroke prevention by percutaneous closure of patent foramen ovale: a systematic review and meta-analysis

Statins in the prevention of dementia and Alzheimer's disease: a meta-analysis of observational studies and an assessment of confounding

Roux-en-Y versus Billroth I reconstruction after distal gastrectomy for gastric cancer: a meta-analysis

A meta-analysis comparing meniscal repair with meniscectomy in the treatment of meniscal tears: the more meniscus, the better outcome?

Combination of radiofrequency ablation with transarterial chemoembolization for hepatocellular carcinoma: a meta-analysis

Extracorporeal membrane oxygenation for severe respiratory failure in adult patients: a systematic review and meta-analysis of current evidence

Statin use and risk of bladder cancer: a meta-analysis

Statin use and risk of kidney cancer: a meta-analysis of observational studies and randomized trials

Postoperative use of the chemopreventive vitamin K2 analog in patients with hepatocellular carcinoma

Aspirin use and risk of age-related macular degeneration: a meta-analysis

Late thrombosis after double versus single drug-eluting stent in the treatment of coronary bifurcations: a meta-analysis of randomized and observational Studies

Should noncurative resection of the primary tumour be performed in patients with stage iv colorectal cancer? A systematic review and meta-analysis

Surgical cytoreduction for recurrent epithelial ovarian cancer

The association between dietary patterns and type 2 diabetes: a systematic review and meta-analysis of cohort studies

Risk of bleeding on triple antithrombotic therapy after percutaneous coronary intervention/ stenting: a systematic review and meta-analysis

A meta-analysis of outcomes of endovascular abdominal aortic aneurysm repair in patients with hostile and friendly neck anatomy

Surgical treatment and survival in patients with liver metastases from neuroendocrine tumors: a meta-analysis of observational studies

Segmentectomy or lobectomy for early stage lung cancer: a meta-analysis

Systematic review and meta-analysis of laparoscopic versus open colectomy with end ileostomy for non-toxic colitis

Risk of cardiovascular disease from antiretroviral therapy for HIV: a systematic review

Reduced esophageal cancer incidence in statin users, particularly with cyclo-oxygenase inhibition

One versus double burr holes for treating chronic subdural hematoma meta-analysis

A meta-analysis of reverse breech extraction to deliver a deeply impacted head during cesarean delivery

Preoperative use of anti-TNF therapy and postoperative complications in inflammatory bowel diseases: a meta-analysis

Meta-analysis of antibiotics and the risk of community-associated Clostridium difficile infection

Laparoscopic right colectomy: a systematic review and meta-analysis of observational studies comparing two types of anastomosis

Infliximab versus cyclosporine as rescue therapy in acute severe steroid-refractory ulcerative colitis: a systematic review and meta-analysis

The Role of Thiopurines in Reducing the Need for Surgical Resection in Crohn's Disease: A Systematic Review and Meta-Analysis

The effect of anesthetic technique on survival in human cancers: a meta-analysis of retrospective and prospective studies

Outcome of Cervicocranial Artery Dissection with Different Treatments: A Systematic Review and Metaanalysis

Sulbactam-based therapy for Acinetobacter baumannii infection: a systematic review and meta-analysis

Cytoreductive surgery and HIPEC for peritoneal metastases combined with curative treatment of colorectal liver metastases: Systematic review of all literature and meta-analysis of observational studies

Cabergoline use and risk of fibrosis and insufficiency of cardiac valves : Meta-analysis of observational studies

Vitamin D treatment and mortality in chronic kidney disease: a systematic review and meta-analysis

The effect of anti-tumor necrosis factor alpha agents on postoperative anastomotic complications in Crohn's disease: a systematic review

Association of sulphonylurea treatment with all-cause and cardiovascular mortality: a systematic review and meta-analysis of observational studies

Comparison of clinical outcome between simultaneous-bilateral and staged-bilateral total knee arthroplasty: a systematic review of retrospective studies

Meta-analysis: colonoscopic post-polypectomy bleeding in patients on continued clopidogrel therapy

Bisphosphonates and risk of subtrochanteric, femoral shaft, and atypical femur fracture: a systematic review and meta-analysis

Use of thiopurines and risk of colorectal neoplasia in patients with inflammatory bowel diseases: a meta-analysis

Antivenom therapy of carpet viper (Echis ocellatus) envenoming: effectiveness and strategies for delivery in West Africa

Laparoscopic total gastrectomy versus open total gastrectomy for cancer: a systematic review and meta-analysis

Systematic review and meta-analysis of robotic surgery compared with conventional laparoscopic and open resections for gastric carcinoma

Effectiveness of Haemophilus influenzae type b vaccines administered according to various schedules: systematic review and meta-analysis of observational data

Safety and efficacy of early pharmacological thromboprophylaxis in traumatic brain injury: systematic review and meta-analysis

Urgent surgery compared with fibrinolytic therapy for the treatment of left-sided prosthetic heart valve thrombosis: a systematic review and meta-analysis of observational studies

The role of statins in prevention and treatment of community acquired pneumonia: a systematic review and meta-analysis

Meta-analysis of clinical outcomes of intravenous recombinant tissue plasminogen activator for acute ischemic stroke: within 3 h versus 3-4.5 h

Anterior corpectomy versus posterior laminoplasty for multilevel cervical myelopathy: a systematic review and meta-analysis

Drugs for attention deficit-hyperactivity disorder do not increase the mid-term risk of sudden death in children: a meta-analysis of observational studies

Laparoscopic distal pancreatectomy and pancreatoduodenectomy: is it worthwhile? A meta-analysis of laparoscopic pancreatectomy

Non-steroidal anti-inflammatory drugs and chronic kidney disease progression: a systematic review

Robot-assisted hysterectomy compared to open and laparoscopic approaches: systematic review and meta-analysis

Staging laparoscopy for the management of early-stage ovarian cancer: a metaanalysis

A Systematic Review and Meta-Analysis of En-Bloc vs Intralesional Resection for Giant Cell Tumor of Bone of the Distal Radius

Antiplatelets versus anticoagulants for the treatment of cervical artery dissection: Bayesian meta-analysis

Does off-pump coronary revascularization reduce mortality in re-operative coronary artery surgery? A meta-analysis of observational studies

Benefits of early tracheotomy: a meta-analysis based on 6 observational studies

Meta-analysis of staged versus combined carotid endarterectomy and coronary artery bypass grafting

The effect of tuberculosis treatment on virologic and CD4+ cell count response to combination antiretroviral therapy: a systematic review

The effect of tuberculosis treatment at combination antiretroviral therapy initiation on subsequent mortality: a systematic review and meta-analysis

Dabigatran for periprocedural anticoagulation following radiofrequency ablation for atrial fibrillation: a meta-analysis of observational studies

Is laparoscopic approach for pancreatic insulinomas safe? Results of a systematic review and meta-analysis

Long-term survival in video-assisted thoracoscopic lobectomy vs open lobectomy in lung-cancer patients: a meta-analysis

Effect of bariatric surgery on oncologic outcomes: a systematic review and meta-analysis

Reduced risk of colorectal cancer with use of oral bisphosphonates: a systematic review and metaanalysis

Laparoendoscopic single-site adrenalectomy versus conventional laparoscopic surgery: a systematic review and meta-analysis of observational studies

A meta-analysis comparing bilateral internal mammary artery with left internal mammary artery for coronary artery bypass grafting

Combined portal vein resection for hilar cholangiocarcinoma: a meta-analysis of comparative studies

Patients get more long-term benefit from central pancreatectomy than distal resection: a meta-analysis

The risk of congenital malformations associated with exposure to beta-blockers early in pregnancy: a meta-analysis

Dose-risk and duration-risk relationships between aspirin and colorectal cancer: a meta-analysis of published cohort studies

Antidepressant use and diabetes mellitus risk: a meta-analysis

Metformin and reduced risk of hepatocellular carcinoma in diabetic patients: a meta-analysis

Use of nonsteroidal anti-inflammatory drugs and bladder cancer risk: a meta-analysis of epidemiologic studies

Meta-analysis of clinical studies comparing coronary artery bypass grafting with percutaneous coronary intervention in patients with end-stage renal disease

Liver Transplantation Versus Liver Resection in the Treatment of Hepatocellular Carcinoma: A Meta-Analysis of Observational Studies

Vitamin D supplementation and mortality risk in chronic kidney disease: a meta-analysis of 20 observational studies

Statin therapy is beneficial for the prevention of atrial fibrillation in patients with coronary artery disease: a meta-analysis

The value of scheduled repeat cranial computed tomography after mild head injury: single-center series and meta-analysis

Endogenous testosterone level and testosterone supplementation therapy in chronic obstructive pulmonary disease (COPD): a systematic review and meta-analysis

Treatment and Management Outcomes of Tibial Eminence Fractures in Pediatric Patients: A Systematic Review

The immunogenicity of anti-TNF therapy in immune-mediated inflammatory diseases: a systematic review of the literature with a meta-analysis

Integrating antiretroviral therapy into antenatal care and maternal and child health settings: a systematic review and meta-analysis

Practical role of mutation analysis for imatinib treatment in patients with advanced gastrointestinal stromal tumors: a meta-analysis

Plasma vitamin D levels, menopause, and risk of breast cancer: dose-response meta-analysis of prospective studies

Crohn's disease and ulcerative colitis are associated with elevated standardized mortality ratios: a meta-analysis

Meta-analysis of plasma to red blood cell ratios and mortality in massive blood transfusions for trauma

Systemic sclerosis (scleroderma) and cancer risk: systematic review and meta-analysis of observational studies

Systematic review and meta-analysis of intraoperative peritoneal lavage for colorectal cancer staging

Iodine and mental development of children 5 years old and under: a systematic review and meta-analysis

Association between QRS duration and outcome with cardiac resynchronization therapy: a systematic review and meta-analysis

Cerebral microbleeds and recurrent stroke risk: systematic review and meta-analysis of prospective ischemic stroke and transient ischemic attack cohorts

Treatment outcomes of treatment-naive Hepatitis C patients co-infected with HIV: a systematic review and meta-analysis of observational cohorts

Thromboembolic events and cardiovascular mortality in inflammatory bowel diseases: A meta-analysis of observational studies

Outcomes after complete versus incomplete revascularization of patients with multivessel coronary artery disease: a meta-analysis of 89,883 patients enrolled in randomized clinical trials and observational studies

The effects of pre-and postoperative fibrinogen levels on blood loss after cardiac surgery: a systematic review and meta-analysis

Association between point-of-care CRP testing and antibiotic prescribing in respiratory tract infections: a systematic review and meta-analysis of primary care studies

Contrast-induced acute kidney injury and risk of adverse clinical outcomes after coronary angiography: a systematic review and meta-analysis

Inflammatory cytokines and risk of coronary heart disease: new prospective study and updated meta-analysis

Carbapenem resistance and mortality in patients with Acinetobacter baumannii infection: systematic review and meta-analysis

Adiponectin and Risk of Stroke: Prospective Study and Meta-analysis

Nonsteroidal anti-inflammatory drugs and risk of ovarian cancer: systematic review and meta-analysis of observational studies

Antidepressant use and new-onset diabetes: a systematic review and meta-analysis

Systematic review of hepatocellular carcinoma mortality rates among hepatitis B virus-infected renal transplant recipients, with supplemental analyses of liver failure and all-cause mortality

Bisphosphonate use and risk of colorectal cancer: a systematic review and meta-analysis

Cancer risk for patients using thiazolidinediones for type 2 diabetes: a meta-analysis

Menopausal hormone therapy and risk of endometrial cancer

Insulin therapy and risk of prostate cancer: a systematic review and meta-analysis of observational studies

Analgesic use and the risk of kidney cancer: A meta-analysis of epidemiologic studies

Fertility treatment and childhood cancer risk: a systematic meta-analysis

Oral contraceptive pills as primary prevention for ovarian cancer: a systematic review and meta-analysis

Use of insulin and insulin analogs and risk of cancer -systematic review and meta-analysis of observational studies

Use of bisphosphonates and the risk of osteonecrosis among cancer patients: a systemic review and meta-analysis of the observational studies

Risk of osteonecrosis in patients taking bisphosphonates for prevention of osteoporosis: a systematic review and meta-analysis

Immunogenicity of monoclonal antibodies against tumor necrosis factor used in chronic immune-mediated Inflammatory conditions: systematic review and meta-analysis

Effects of asthma severity, exacerbations and oral corticosteroids on perinatal outcomes

Consumption of omega-3 fatty acids and the risk of skin cancers: A systematic review and meta-analysis

Statins use and risk of depression: A systematic review and meta-analysis

Statins and primary liver cancer: a meta-analysis of observational studies

Reproductive and exogenous hormone factors in relation to risk of meningioma in women: a meta-analysis

Oral contraceptive use and uterine leiomyoma risk: a meta-analysis based on cohort and case-control studies

Clinical risk factors of delayed gastric emptying in patients after pancreaticoduodenectomy: a systematic review and meta-analysis

Relationship between use of antidepressants and risk of fractures: a meta-analysis

Rheumatoid factor and response to TNF antagonists in rheumatoid arthritis: Systematic review and meta-analysis of observational studies

Statin use and the risk of biochemical recurrence of prostate cancer after definitive local therapy: a meta-analysis of eight cohort studies

The protective effect of alcohol on developing rheumatoid arthritis: a systematic review and meta-analysis

Preoperative statin therapy is associated with lower requirement of renal replacement therapy in patients undergoing cardiac surgery: a meta-analysis of observational studies

Statins are associated with reduced risk of gastric cancer: a systematic review and meta-analysis

Acid-suppressive medications and risk of oesophageal adenocarcinoma in patients with Barrett's oesophagus: a systematic review and meta-analysis

Statins are associated with reduced risk of esophageal cancer, particularly in patients with Barrett's esophagus: a systematic review and meta-analysis

Statins are associated with a reduced risk of hepatocellular cancer: a systematic review and meta-analysis

Association of statin use with risk of dementia: a meta-analysis of prospective cohort studies

Insulin therapy and risk of colorectal cancer: an updated meta-analysis of epidemiological studies

Bisphosphonate treatment and risk of esophageal cancer: a meta-analysis of observational studies

The association between histamine 2 receptor antagonist use and Clostridium difficile infection: a systematic review and meta-analysis

beta-adrenergic receptor antagonists and fracture risk: a meta-analysis of selectivity, gender, and site-specific effects

Does early life exposure to antibiotics increase the risk of eczema? A systematic review

Adverse peri-operative outcomes following elective total hip replacement in diabetes mellitus: a systematic review and meta-analysis of cohort studies

Statin use and risk of Parkinson's disease: a meta-analysis of observational studies

Myocardial infarction and individual nonsteroidal anti-inflammatory drugs meta-analysis of observational studies

A systematic review and meta-analysis on primary percutaneous coronary intervention of an unprotected left main coronary artery culprit lesion in the setting of acute myocardial infarction

Insulin therapy contributes to the increased risk of colorectal cancer in diabetes patients: a meta-analysis

Flavonol intake and the risk for stroke: A meta-analysis of cohort studies

Tricyclic antidepressant use and risk of fractures: a meta-analysis of cohort and case-control studies

Statins are associated with reduced risk of gastric cancer: a meta-analysis

Isoflavone consumption and risk of breast cancer: a dose-response meta-analysis of observational studies

Association between use of benzodiazepines and risk of fractures: a meta-analysis

Acid suppressive drugs and gastric cancer: a meta-analysis of observational studies

Acid-suppressive therapy is associated with spontaneous bacterial peritonitis in cirrhotic patients: a meta-analysis

Vitamin D intake and risk of type 1 diabetes: a meta-analysis of observational studies

Pregnancy outcome following loop electrosurgical excision procedure (LEEP) a systematic review and meta-analysis

Aspirin and non-aspirin NSAIDs increase risk of colonic diverticular bleeding: a systematic review and meta-analysis

Effects of percutaneous revascularization of chronic total occlusions on clinical outcomes: a meta-analysis comparing successful versus failed percutaneous intervention for chronic total occlusion

Anaemia, prenatal iron use, and risk of adverse pregnancy outcomes: systematic review and meta-analysis

Proton pump inhibitors and the risk of hospitalisation for community-acquired pneumonia: replicated cohort studies with meta-analysis

Early application of airway pressure release ventilation may reduce mortality in high-risk trauma patients: a systematic review of observational trauma ARDS literature

Meta-analysis of joint preservation versus arthroplasty for the treatment of displaced 3-and 4-part fractures of the proximal humerus

Systematic review of clofazimine for the treatment of drug-resistant tuberculosis

Effect of oral appliances on blood pressure in obstructive sleep apnea: a systematic review and meta-analysis

Effects of deep brain stimulation in dyskinetic cerebral palsy: a meta-analysis

Rasburicase in tumor lysis syndrome of the adult: a systematic review and meta-analysis

A systematic review and meta-analysis of hybrid aortic arch replacement

Effects of monitoring strategies on seizures in pregnant women on lamotrigine: a meta-analysis

Estrogen and selective estrogen receptor modulators (SERMs) for the treatment of acromegaly: a metaanalysis of published observational studies

Survival impact of complete cytoreduction to no gross residual disease for advanced-stage ovarian cancer: a meta-analysis

Effectiveness of seasonal influenza vaccines in children -a systematic review and meta-analysis

Different combined oral contraceptives and the risk of venous thrombosis: systematic review and network meta-analysis

Strategies for reducing treatment default in drug-resistant tuberculosis: systematic review and metaanalysis

Efficacy of adoptive immunotherapy with donor lymphocyte infusion in relapsed lymphoid malignancies

Meta-analysis: the safety and efficacy of dilation in eosinophilic oesophagitis

Prevalence of complications in neuromuscular scoliosis surgery: a literature meta-analysis from the past 15 years

A systematic review and meta-analysis on the safety and efficacy of the frozen elephant trunk technique in aortic arch surgery

Long-term Outcome of One-Piece Implants. Part II: Prosthetic Outcomes. A Systematic Literature Review with Meta-Analysis

Determining the noninfectious complications of indwelling urethral catheters: a systematic review and meta-analysis

Prevalence and factors associated with glucocorticoids (GC) use in systemic sclerosis (SSc): a systematic review and meta-analysis of cohort studies and registries

Efficacy and durability of radiofrequency ablation for Barrett's Esophagus: systematic review and meta-analysis

Biologic grafts for ventral hernia repair: a systematic review

A meta-analysis of observational intra-arterial stroke therapy studies using the Merci device, Penumbra system, and retrievable stents

Use of combination neonatal prophylaxis for the prevention of mother-to-child transmission of HIV infection in European high-risk infants

A meta-analysis of operative versus nonoperative treatment in 463 scapular neck fractures

The role of postoperative radiation and chemoradiation in merkel cell carcinoma: a systematic review of the literature

Meta-analysis of three observational studies of amlodipine/valsartan in hypertensive patients with additional risk factors

Can statins improve outcomes after isolated cardiac valve surgery? A systematic literature review

Combined anticoagulation and antiplatelet therapy for high-risk patients with atrial fibrillation: a systematic review

Comparative effectiveness of interventions for children exposed to nonrelational traumatic events

Do children with uncomplicated severe acute malnutrition need antibiotics? A systematic review and meta-analysis

Does the application of incisional negative pressure therapy to high-risk wounds prevent surgical site complications? A systematic review

Stress ulcer prophylaxis in the intensive care unit: is it indicated? A topical systematic review

Cardiotoxicity in cancer patients treated with 5-fluorouracil or capecitabine: a systematic review of incidence, manifestations and predisposing factors

Can Cardiac Resynchronization Therapy Improve Cognitive Function? A Systematic Review

Antipsychotics: a real or confounding risk factor for venous thromboembolism?

Complications of pelvic organ prolapse surgery and methods of prevention

Adverse effects of the common treatments for polycystic ovary syndrome: a systematic review and meta-analysis

Effect of influenza vaccination of healthcare personnel on morbidity and mortality among patients: systematic review and grading of evidence

Fixed-dose combination antituberculosis therapy: a systematic review and meta-analysis

Effectiveness and harms of recombinant human bone morphogenetic protein-2 in spine fusion: a systematic review and meta-analysis

Treatment of anemia in patients with heart disease: a systematic review

Nutritional interventions for reducing gastrointestinal toxicity in adults undergoing radical pelvic radiotherapy

Therapy of Endocrine Disease: Impact of iodine supplementation in mild-to-moderate iodine deficiency: systematic review and meta-analysis

Low-versus high-dose rituximab for rheumatoid arthritis: A systematic review and meta-analysis

The use of postoperative topical corticosteroids in chronic rhinosinusitis with nasal polyps: a systematic review and meta-analysis

No consistent evidence of differential cardiovascular risk amongst proton-pump inhibitors when used with clopidogrel: meta-analysis

Risk of liver toxicity with the angiogenesis inhibitor pazopanib in cancer patients

Sexual Dysfunction associated with Second-Generation Antidepressants in Patients with Major Depressive Disorder: Results from a Systematic Review with Network Meta-Analysis

Vitamin and Mineral Supplements in the Primary Prevention of Cardiovascular Disease and Cancer: An Updated Systematic Evidence Review for the U.S. Preventive Services Task Force

Endoscopic sphincterotomy plus balloon dilation versus endoscopic sphincterotomy for choledocholithiasis: A meta-analysis

Association between statin use and colorectal cancer risk: a meta-analysis of 42 studies

A meta-analysis of self-administered vs directly observed therapy effect on microbiologic failure, relapse, and acquired drug resistance in tuberculosis patients

Triple antithrombotic therapy versus double antiplatelet therapy after percutaneous coronary intervention with stent implantation in patients requiring chronic oral anticoagulation: a meta-analysis

Comparing minimally invasive and open transforaminal lumbar interbody fusion for treatment of degenerative lumbar disease: a meta-analysis

Transcatheter aortic valve implantation versus surgical aortic valve replacement for severe aortic stenosis: a meta analysis

Statins and amyotrophic lateral sclerosis: a systematic review and meta-analysis

The Effect of Flapless Surgery on Implant Survival and Marginal Bone Level: A Systematic Review and Meta-analysis

Association of inhibitors of gastric acid secretion and higher incidence of necrotizing enterocolitis in preterm very low-birth-weight infants

Does tranexamic acid stop haemoptysis?

Thirty-day mortality after coronary artery bypass surgery in patients aged <50 years: results of a multicenter study and meta-analysis of the literature

Isotretinoin and risk for inflammatory bowel disease: a nested case-control study and meta-analysis of published and unpublished data

Contralateral carotid occlusion in endovascular and surgical carotid revascularization: a single centre experience with literature review and meta-analysis

A trial of mercaptopurine is a safe strategy in patients with inflammatory bowel disease intolerant to azathioprine: an observational study, systematic review and meta-analysis

Does BCG vaccination protect against childhood asthma? Final results from the Manchester Community Asthma Study retrospective cohort study and updated systematic review and meta-analysis

Prospective cohort study and meta-analysis of cyclic bleeding after laparoscopic supracervical hysterectomy

Vaccines for preventing influenza in people with asthma

Antiplatelet Treatment for Prevention of Cerebrovascular Events in Patients With Vascular Diseases: A Systematic Review and Meta-Analysis

Biphasic versus monophasic defibrillation in out-of-hospital cardiac arrest: a systematic review and meta-analysis

A meta-analysis of transcatheter closure of patent foramen ovale versus medical therapy for prevention of recurrent thromboembolic events in patients with cryptogenic cerebrovascular events

Observational studies in the era of randomized trials: finding the balance

Challenges in systematic reviews that assess treatment harms

Observational studies are complementary to randomized controlled trials

Observational studies: going beyond the boundaries of randomized controlled trials

Why we need observational studies to evaluate the effectiveness of health care

A comparison of observational studies and randomized, controlled trials

Observational study designs for comparative effectiveness research: an alternative approach to close evidence gaps in head-and-neck cancer

From randomized controlled trials to observational studies

Randomized clinical trials and observational studies: guidelines for assessing respective strengths and limitations

Revisiting issues, drawbacks and opportunities with observational studies in comparative effectiveness research

Potential bias of instrumental variable analyses for observational comparative effectiveness research

The role of observational investigations in comparative effectiveness research. Value Health

Quality standards for real-world research. Focus on observational database studies of comparative effectiveness

Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles

Observational studies in systematic [corrected] reviews of comparative effectiveness: AHRQ and the Effective Health Care Program

Chronic Subdural Hematoma Management: A Systematic Review and Meta-analysis of 34829 Patients

Systematic review and meta-analysis of perioperative and oncological outcomes of laparoscopic cryoablation versus laparoscopic partial nephrectomy for the treatment of small renal tumors

Macrolides and Mortality in Critically Ill Patients With Community-Acquired Pneumonia: A Systematic Review and Meta-Analysis

Liberal Versus Restricted Fluid Resuscitation Strategies in Trauma Patients: A Systematic Review and Meta-Analysis of Randomized Controlled Trials and Observational Studies

Safety and effectiveness of recombinant human bone morphogenetic protein-2 for spinal fusion: a meta-analysis of individual-participant data

Risk of fracture with thiazolidinediones: an individual patient data meta-analysis

A first look at the volume and cost of comparative effectiveness research in the United States

Meta-analysis of welldesigned nonrandomized comparative studies of surgical procedures is as good as randomized controlled trials

Randomized, controlled trials, observational studies, and the hierarchy of research designs

Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview

Comparison of evidence of treatment effects in randomized and nonrandomized studies

When are observational studies as credible as randomised trials?

Observational studies using propensity score analysis underestimated the effect sizes in critical care medicine

Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated

The Cochrane Collaboration

Methodology and reporting of systematic reviews and meta-analyses of observational studies in psychiatric epidemiology: systematic review

Room for improvement? A survey of the methods used in systematic reviews of adverse effects

Epidemiology and reporting characteristics of systematic reviews

Quality of reporting in systematic reviews of adverse events: systematic review

Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement

Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews

Participation of epidemiologists and/or biostatisticians and methodological quality of published controlled clinical trials

Can trial quality be reliably assessed from published reports of cancer trials: evaluation of risk of bias assessments in systematic reviews

How to read a systematic review and meta-analysis and apply the results to patient care: users' guides to the medical literature

Making prospective registration of observational research a reality

Public availability of results of observational studies evaluating an intervention registered at ClinicalTrials.gov

The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey

In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias

Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study

The Cochrane Collaboration's tool for assessing risk of bias in randomised trials

Assessing the quality of reports of randomized clinical trials: is blinding necessary?

Grading quality of evidence and strength of recommendations

The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses

Philippe Ravaud is director of the French EQUATOR Centre and a member of the EQUATOR Network Steering Group.

The researchers did not receive external sources of funding.

The authors declare that they have no competing interests.Authors' contribution TF was involved in the study conception, study selection, data extraction, interpretation of results, and drafting the manuscript. PR was involved in the study conception, interpretation of the results, and drafting the manuscript. CR was involved in data extraction. EP was involved in data analysis. AD was involved in the study conception, interpretation of the results, and drafting the manuscript. All authors read and approved the final manuscript.• We accept pre-submission inquiries • Our selector tool helps you to find the most relevant journal Submit your next manuscript to BioMed Central and we will help you at every step: