I read with much interest the Review by Hao et al. (Traditional Chinese medication for cardiovascular disease. Nat. Rev. Cardiol. 12, 115–122; 2015).1 The authors reviewed the published randomized, controlled trials (RCTs) of traditional Chinese medicine (TCM) for the treatment of cardiovascular diseases (CVDs). The Review has attracted a lot of attention among those familiar with the literature on TCM, owing to the application of an evidence-based approach to this topic.

Since 2002, much research has been performed into the potential benefits, harm, and drug interactions of herbal medicine in CVD,2,3 including an expert consensus document on complementary and integrative medicine released by the ACC.4 Currently, a resurgence of scientific interest is occurring into the use of TCM in Western countries among patients with CVDs.5 Given the limitations of traditional, single-target therapy based on the 'one-target one-disease' model in the treatment of complex diseases, including CVDs and cancer, scientists have turned their attention to TCM because of its holistic approach and multitarget therapeutics.6 As stated by the authors of the Review, “TCM treatment was most likely to be used by patients with a long history of coronary heart disease or with a history of stroke”.1 However, in my opinion, no critically appraised evidence on the efficacy and safety of TCM exists to justify its clinical use and recommendation.

I appreciate the tremendous work by the authors, who aimed to assess the current clinical evidence for the use of TCM in patients with hypertension, coronary heart disease, cardiac arrhythmias, or heart failure. They attached great importance to the definite diagnosis of CVDs, sample size, methodological quality (Jadad score ≥2), treatment duration, hard and surrogate end points, and adverse cardiovascular events in all the trials selected for inclusion in the Review. However, I am disappointed that many serious drawbacks existed, mainly in the search strategy, literature citation, and results reporting.

Firstly, the major impediment to evaluating the efficacy of TCM was the insufficient literature retrieval. Only 68 studies were found to be eligible for inclusion in the Review, probably because of technical difficulties and ambiguous selection criteria. According to the definition of TCM, the term refers to a series of traditional medical practices that originated in China, including Chinese herbal medicine, dietary supplements, acupuncture, moxibustion, massage, cupping therapy, qigong, Tai Chi, and other exercise therapy.7 Although no formal restriction was imposed on the types of TCM intervention that could be considered in the Review, only Chinese herbal medicine was included. If this restriction was a formal selection criterion, it should have been stated in the 'Search strategy and selection criteria' section. For example, many rigorously designed RCTs with high-quality data on the use of acupuncture,8 qigong,9 and other therapies beyond Chinese herbal medicine for the treatment of hypertension have been published in English during the past 10 years. Moreover, numerous appropriate studies on Chinese herbal medicine for the treatment of hypertension published in either English or Chinese were omitted.10,11

I am also concerned that the manual retrieval of articles using the search terms listed was inadequate to identify all relevant RCTs into TCM for CVDs. Some large-scale, multicentre, double-blind, placebo-controlled RCTs with hard end points for coronary heart disease were not included in the Review, such as investigations into the use of Qi-Shen-Yi-Qi dripping pills for the secondary prevention of myocardial infarction,12 and the use of combination therapy for patients with acute coronary syndrome after percutaneous coronary intervention.13 According to the Review criteria, the authors focused primarily on studies published after 1 January 1998. I believe that arbitrarily excluding eligible RCTs published before 1998 is unreasonable. For example, in the first report of a multicentre RCT in the field of TCM from 1982, Chen and colleagues evaluated the effect of Jingzhiguanxin tablets on surrogate end points in treating angina pectoris.14 Therefore, I suggest that no restriction on publication date should have been set, and all the electronic databases should have been searched from their inception up to 2014, to avoid potential selective bias. Furthermore, given that TCM is mainly practised and researched in China, I believe that the Wanfang database (in addition to the other three main Chinese electronic databases—the China National Knowledge Internet, the China Biology Medicine Database, and the VIP database) should have been searched to identify the maximum number of clinical trials. Owing to these major drawbacks in the literature searching, a large number of well-designed RCTs were omitted, which greatly weakens the authors' conclusions. Therefore, I suggest that experts in both TCM and cardiovascular medicine should have been involved in researching data for the Review.

Secondly, serious drawbacks regarding results reporting were also identified. Although the authors tried to evaluate the evidence for the use of TCM for CVDs based on the included RCTs, the conclusions they reached seem to be biased. For example, in the section on hypertension, a total of 10 trials involving 1,658 patients were included. The therapeutic effects of TCM were compared with no intervention (references 19–24) or Western medications (references 25–28). The authors conclude that “compared with blank control or placebo, TCM was efficacious in lowering blood pressure in patients with hypertension” and “compared with Western medication, TCM had similar therapeutic effects in patients with hypertension”.1 In my opinion, however, these findings need to be treated with caution.

No definitive double-blind, randomized, placebo-controlled trials that were particularly well designed to evaluate the efficacy of TCM for hypertension were identified in this Review. According to the Cochrane collaboration tool, all the included trials were assessed as having a high or unclear risk of bias owing to inadequate reporting of the study design, random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, intention-to-treat analysis, or drop-outs. Additionally, either an 'A + B versus B' or an 'A versus B' study design was utilized in the trials. That is, patients with hypertension were randomly allocated to receive TCM plus Western medication, TCM only, or Western medication only; no placebo controls were applied. TCM is known to have a strong placebo effect—without a rigorous placebo control, these study designs might exaggerate the antihypertensive effects of TCM. Moreover, in an analysis using the Revman 5.3 software, significant heterogeneity in the data was found to exist between studies on systolic and diastolic blood pressure, with I2 ranging from 72% to 99%.

Considering the poor methodological design, small sample size, limited numbers of included trials, and the high heterogeneity, no valid evidence regarding the blood-pressure-lowering effect of TCM was identified. Therefore, in my opinion, the statements that “TCM might have moderate antihypertensive effects”, “TCM was efficacious in lowering blood pressure”, and “TCM had similar therapeutic effects [to Western medication]”1 are not evidence-based. Furthermore, given that adverse effects were reported in only four trials (40%), the safety of TCM is still unknown. Therefore, the conclusion that “[TCM] seems to be well tolerated”1 is not appropriate. Owing to insufficient clinical data, drawing a definite conclusion about the effectiveness and safety of TCM for treatment of hypertension is difficult. Similar conclusions about the use of TCM in hypertension were also reached in our numerous evidence-based reviews.15,16

In conclusion, the Review by Hao et al. is an important step forwards in evaluating the role of TCM for the treatment of CVDs. However, the Review raised considerable concerns about the insufficient literature search and the flawed original RCTs. Therefore, the reported strength of evidence should be re-evaluated, and more rigorously-designed, placebo-controlled trials are warranted to give high-level evidence in future studies.