key: cord-0009556-zssy3i4d
authors: Wang, Junwen; Cui, Meng; Jiao, Hongguan; Tong, Yuanyuan; Xu, Jun; Zhao, Yingkai; Han, Mei; Liu, Jianping
title: Content analysis of systematic reviews on the effectiveness of Traditional Chinese Medicine
date: 2013-09-13
journal: nan
DOI: 10.1016/s0254-6272(13)60118-1
sha: 0d3d733404ffc141f10dcd7d41efdaa6103ec5b2
doc_id: 9556
cord_uid: zssy3i4d

OBJECTIVE: To evaluate evidence for the efficacy of Traditional Chinese Medicine (TCM) in systematic reviews. METHODS: Chinese (TCMPeriodical Literature Database, Chinese Biological Medicine database, Chinese Medical Current Contents, China Hospital Knowledge Database journal fulltext database, Virtual Machining and Inspection System, and Wanfang) and English (Cochrane Database of Systematic Reviews, PubMed and Embase) databases were searched. RESULTS: Three thousand, nine hundred and fifty-five articles were initially identified, 606 of which met the inclusion criteria, including 251 in English (83 from the Cochrane Database) and 355 in Chinese. The number of articles published each year increased between 1989 and 2009. Cardiocerebrovascular disease was the most studied target disease. Intervention measures includedTCM preparations (177 articles), acupuncture (133 articles) and combinations of TCM and western medicine (38 articles). Control measures included positive medical (177 articles), basic treatment (100 articles), placebo (219 articles), and blank and mutual (107 articles). All articles included at least one reference; the greatest number was 268. Six of 10 articles with high quality references demonstrated curative effectsagainst target diseasesincludingupper respiratory tract infection, dementia and depression. Interventions that were not recommendedwere tripterygium for rheumatoid arthritis andTCM syndrome differentiation for pediatric nocturia. In 10.4% of the studies, the authors concluded that the intervention had a curative effect. The assessors agreed with the authors' conclusions in 88.32% of cases, but rejected 8.94% (54 articles). CONCLUSION: 1) Training in systematic review methods, including topic selection, study design, methods and technology, should be improved. 2) Upper respiratory tract infection, dementia and depression may become the predominant diseases treatedby TCM, and the corresponding interventions could be developed into practical applications. 3) Use of non-recommended interventions should be controlled, and there should be more research on side effects.

Intervention measures included TCM preparations (177 articles), acupuncture (133 articles) and combinations of TCM and Western Medicine (38 articles). Control measures included positive medical (177 articles), basic treatment (100 articles), placebo (219 articles), and blank and mutual (107 articles). All articles included at least one reference; the greatest number was 268. Six of 10 articles with high quality references demonstrated curative effectsagainst target diseases including upper respiratory tract infection, dementia and depression. Interventions that were not recommended were tripterygium for rheumatoid arthritis and TCM syndrome differentiation for pediatric nocturia. In 10.4% of the studies, the authors concluded that the intervention had a curative effect. The assessors agreed with the authors' conclusions in 88.32% of cases, but rejected 8.94% (54 articles).

CONCLUSION: 1) Training in systematic review methods, including topic selection, study design, methods and technology, should be improved. 2) Upper respiratory tract infection, dementia and depression may become the predominant diseases treated by TCM, and the corresponding interventions could be developed into practical applications. 3) Use of non-recommended interventions should be controlled, and there should be more research on side effects.

Evidence-based medicine is influential worldwide and has led to new requirements in the field of Traditional Chinese Medicine (TCM). TCM professionals have conformed to this trend and have conducted many studiesguided by the concepts and methods of evidence-based medicine. However, existing systematic reviews of TCM have found no evidence of its effectiveness, leading to confusion. Guo et al 1 suggested that use of a therapy without evidence of its effectiveness is not recommended, because of potential adverse reactions and negative herb-herb and herb-chemical medicine interactions. However, Pittilo et al 2 considered such negative views inappropriate, being based on the few inclusions trials. Lack of evidence does not mean non-existence of evidence. Many problems remain; for example, what can be gained from existing systematic reviews? Is there any clearly confirmed or refuted evidence of efficacy in systematic reviews of the effectiveness of TCM? This study aimed to collect systematic reviews of TCM and undertake content analysis to help solve these problems.

Inclusion criteriawere as follows: 1) The use of systematic evaluation, systematic summaries or meta-analysis could be confirmed by reading the document's bibliography or contents. 2) Target intervention measures involved TCM, including Chinese herbal medicine, traditional Chinese herbs and their preparations, acupuncture (including acupuncture, acusector, auricular acupuncture and acupoint) and combinations of TCM and Western Medicine. Articles were excluded based on their systematic review scheme or quality, and whether they had been published more than once (with the exception of articles published in both Chinese and English).

The Chinese language data retrieval sourcessearched in this study were the TCM Periodical Literature Database, the Chinese Biological Medicine Database, Chinese Medical Current Contents, the China Hospital Knowledge Database journal fulltext database, the Virtual Machining and Inspection System and the Wanfang database. English language sources were the Cochrane Database of Systematic Reviews (Cochrane Reviews), PubMed and Embase. Systematic reviews were searched for by computerand retrieved after the articles had been read. Article searching and screening were performed by two different reviewers. Both screening and collection results were recorded. The searcher was qualified to undertake data searches using the latest methods and determined the retrieval strategy independently (asking experts for advice if unsure). Disagreements during article screening were resolved by discussion with experts.

Data were extracted and checked by two reviewers independently. Any disagreement on data extraction or evaluation was resolved by discussion. The following data were extracted from every included study. Basic information: publication information was obtainedautomatically. Number of articles published each year, research institution, location of first author, and English and Chinese journals that published systematic reviews of TCM effectiveness were described by frequency statistics. Content analysis of relevant data: 1) Initial data on target diseases, intervention measures, control measures, final results index and secondary classification. 2) Information on the research, including number of included studies, number of highquality studies, forest graphs and funnel plots, and whether sensitivity analysis was needed. 3) Auxiliary information on the quality of the studies evaluated according to evidence-based medicine methodology, including number of authors and databases searched. 4) For systematic reviews that included a high quality randomized control trial, which was extracted as the best evidence for archiving.

Our search of the electronic databases identified 3955 articles for review. After reading the titles and abstracts and eliminating animal experiments, general reviews and target intervention measures unrelated to Chinese medicine, we obtained 805 fulltext article abstracts. Eighty-three were not further evaluated; 10 of these were in German, Spanish or Japanese, three were Cochrane articles published in 2010, 26 were other English articles and 44 were Chinese articles. The remaining 280 articles in English (83 Cochrane) and 442 in Chinese were collected in fulltext form. One hundred and sixteen articles were excluded after fulltext checkingdue to their quality or because their systematic assessment was unrelated to treatment. Finally, 606 articles that met the inclusion criteria were obtained, comprising 251 in English (including 83 Cochrane articles) and 355 in Chinese ( Figure 1 ). Anotherfourarticles [3] [4] [5] [6] were found by searching the included articles' references. The references of these four articles were included in the study. 7 the first in Chinesewas in 1998. 8 From then, the number increased year on year. Journals publishing systematic reviews on TCM effectiveness: the top 10 journals publishing systematic reviews of the effectiveness of TCM are listed in Table 1 . Professional evidence-based medical tools-the Cochrane Database of Systematic Reviews and Zhong Guo Xun Zheng Yi Xue Za Zhi-were the first ranked English and Chinese journals, respectively.

English articles: The top 10 locations of the first au- Table 2 . Sichuan, Beijing and Hong Kong in China were ranked second, fifth and sixth, respectively; other locations were, in descending order, UK, USA, Australia, Canada, the Republic of Korea, Germany and the Netherlands. The top 10 institutions of the first authors of the English articles are listed in Table 3 and included three Chinese centers: the West China Hospital, Beijing University of Chinese Medicine and Hong Kong Baptist University, which ranked first, fourth and fifth, respectively. Chinese articles: the top 10 locations of the first authors of the Chinese articles are displayed in Table 4 . Most were in Guangdong, Sichuan, Beijing, Tianjin or Shandong, each of which accounted for more than 10% of the total number of articles. The top 10 institutions of the first authors of the Chinese articles are listed in Table 5 and including seven professional TCM research institutions. Sichuan University, Fudan University and Lanzhou University, which are not TCM centers, were ranked fourth and joint tenth, respectively.

Studied diseases: in general, cardiocerebrovascular diseases were the most common targetsin studies of the effectiveness of TCM, including 32 studies on ischemic stroke, 11 on acute cerebral infarction and 11 on angina pectoris (Table 6 ). Post-stroke syndrome was mostly treated by acupuncture. Interventions: fifty-six articles focused on TCM interventions (18 Chinese herbal medicine, 22 TCM and 16 Chinese medicine), accounting for 9.2% of the studies. One hundred and thirty-three articles focused on acupuncture (21.9% ), 21 on pinprick (3.5% ), six on acusector (1.0% ), four on auricular acupuncture (0.7%) and three on massage (0.5%). Thirty-eight articles focused on combinations of TCM and Western Medicine (6.3% ) and twoon treatment based on syndrome differentiation (0.3%). Among the 606 articles, nine did not have a control group and 64 did not report the control group intervention measure. One hundred and seventy-seven articles used positive medical controls (86 Western Medicine, 51 Chinese medicine plus Western Medicine and 40 Chinese Medicine). Four used sham acupuncture, rehabilitation oracupoint injection plus massage as controls, 100 used basic treatment, 219 had placebo controls and 107 had blank and mutual controls (in the evaluation of stroke).

The smallest number of drugs or placeboused in the control groups was one; the most were six Chinese medical treatments and eight western. Some studies used various combinations of controls simultaneously, such as acupuncture plus Chinese medicine plus physical therapy plus Western Medicine or Chinese medicine plus Western Medicine plus placebo. Number of included studies: all of the 606 articles included at least one reference; the largest number was 268. Two hundred and ninety-five articles included one to 10 references, 244 included 10-30 and 67 included more than 30. Thirty-four articles included controlled clinical trials, 18 included references without controls and two were unknown. Effectiveness indices: only 45 of the 606 articles used a gold standard criterion as an index of effectiveness (7.43%). Two hundred and twelve articles used efficiency as the outcome index. Two hundred and ninety-three articles used several indices and 56 use do the standards. Forest graphs: two hundred and eighty-two articles had no forest graphs (46.5% ). Forest graphs showed effectiveness of the treatment measure in 166 articles (27.4%), invalid results in 44 (7.3%) and uncertain results in 17 (2.8%). The other 96 articles (15.8%) contained several forest graphs indicating in consistent results.

Authors' conclusions: in 10.4% of articles, the authors were very sure about the curative effect of the evaluated TCM for a particular disease; 39.1% of authors thought it was effective, but not confirmed due to poor quality of the included research or low strength of evidence; 47.0% of authors had a neutral attitude, neither confirming nor denying the evaluated TCM's effectiveness, and considered further research necessary; 2.5% thought the study conclusions were invalid anddid not recommend the treatment based on safety considerations; and 0.7% strongly suggested prohibition of the studied intervention. Six of 10 articles with high quality references demonstrated curative effects on target diseases including acute upper respiratory tract infection, upper respiratory tract infection, dementia, depression, functional indigestion and outer upper ankle pain. The authors of two articles thought that there was no obvious difference in curative effect between the experimental group and the control group, but they did not recommend the TCM intervention due to possible side effects. The target intervention and diseases in these two studies were tripterygium for rheumatoid arthritis and syndrome differentiation treatment for pediatric nocturia. Another two articles demonstrated effectiveness, but the authors suggested further studies due to the small number of included articles and potential bias (Table 7) . Assessors'conclusions: the assessors' approved the authors' conclusionsin 88.32% of the articles; in 8.94% (54 articles), the conclusions were rejected. Of these, 19 articles failed due to the study being too simplistic and the loss of key materials, 16 made biased conclusions based on very low quality documents, fourhad obvious mistakes in meta-analysis forest graphs, threeused self-contradictory TCM control settings, and 12 made 

This study identified 606 systematic summaries or meta-analyses of TCM published during 1989-2009 domestically and abroad. The number of articles published each year showed an upward trend, which indicates that the international medical field has a growing interest in clinical researchon TCM and the demand for evidence of its effectiveness is increasing.

To aid TCM clinical practice and research, we propose suggestions for the conduct of future systematic reviews.

The number of domestic systematic reviews andmeta-analyses is increasing year on year, but some are of poor quality. The present study suggests that formal training is necessary for researchers engaged in the conduct of systematic summaries and manuscript preparation. In addition, the following aspects of systematic review should be fully understood and specified. Topic selection and study design in systematic reviews differ from traditional methods. They should include detailed objectives, target diseases, intervention measures, control measures and clear, specific results indices. For example, TCM is not appropriate as an intervention measure when evaluating the treatment effect of TCM on a certain disease, because there is a large variability and bias will be increased in the evaluation. Regarding choice of results index, 32.12% of systematic review articles in the TCM field used efficiency as the curative effect standard due to a lack or deficiency of gold standards in Chinese medicine diagnosis and classification. However, the definition of efficiency differs between articles and thus an index for the curative effect of TCM is required.

Systematic review methodology and techniques should conform to the Cochrane Systematic Review manual, which gives detailed instructions for article retrieval, random and non-random quality evaluation, material integration (meta-analysis), explanation of results and reportwriting methods. Sources of data should be comprehensive, so searching fewer than three databases or only Chinese databases is not appropriate. A quality evaluation tool must be used to assess the quality and characteristics of the included studies. Preferably, the results of systematic reviews should be reported according to the Quality of Reporting of Meta-analyses statement. It would be desirable if clinical workers, methodologists and evidence-based medical workers participate jointly in the quality evaluation of clinical research articles, and in the selection of statistical methods and control and intervention measures.

The target diseases of studies with high level evidence in this research included acute upper respiratory tract infection, upper respiratory tract infection, dementia, depression, functional indigestion and outer upper ankle pain. These may become the predominant diseases treated with TCM, so the appropriate interventions could be developed into practical applications. The target intervention measures and diseases for which there was negative evidence-that is, the evidence suggested they should not be recommendedwere tripterygium treatment for rheumatoid arthritis and syndrome differentiation treatment for pediatric nocturia. There are two reasons why these therapies were not recommended: uncertainty of curative effect and potential side effects. Research into the side effects of such interventions should be increased, because this may facilitate their future use in practice.

A systematic review of randomised clinical trials of individualised herbal medicine in any indication

Report to ministers from the department of health steering group on the statutory regulation of practitioners of acupuncture, herbal medicine, Traditional Chinese Medicine and other traditional medicine systems practiced in the UK

Quality appraisal of systematic reviews or meta-analysis on Traditional Chinese Medicine published in Chinese journals

Systematic review of acupuncture published 1996-2005

Acupuncture resources in cochrane database of systematic reviews

Assessment of systematic reviews and meta-analyses on Traditional Chinese Medicine published in Chinese journals. Zhong Guo Xun-Zheng Yi Xue Za Zhi

A meta-analysis of acupuncture for chronic pain

Meta-analysis of Chinese herbs for esophagus precancerous lesion. Shi Yong Zhong Xi Yi Jie He

Chinese medicine for acute upper respiratory tract infection: a systematic review of randomized controlled trials

Herbal medicine for dementia: a systematic review

Is acupuncture beneficial in depression: a meta-analysis of 8 randomized controlled trials

Acupuncture for pelvic and back pain in pregnancy: a systematic review

Acupuncture for the alleviation of lateral epicondyle pain: a systematic review

Andrographispaniculata in the treatment of upper respiratory tract infections: a systematic review of safety and efficacy

A systematic review of randomised clinical trials of tripterygiumwilfordii for rheumatoid arthritis

A systematic review of randomised clinical trials of individualised herbal medicine in any indication

Chinese and Western Medicine for treatment of functional dyspepsia: a systematic review of randomized controlled trials

Acupuncture for tension-type headache: a meta-analysis of randomized, controlled trials