Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Assessing the Quality of Reports about Randomized Controlled Trials of Acupuncture Treatment on Mild Cognitive Impairment

  • Xiao Lu,

    Affiliations Tianjin University of Traditional Chinese Medicine, Tianjin, China, MOE Virtual Research Center of Evidence-Based Medicine, Chengdu, China

  • Shang Hongcai ,

    shanghongcai@foxmail.com

    Affiliations Tianjin University of Traditional Chinese Medicine, Tianjin, China, MOE Virtual Research Center of Evidence-Based Medicine, Chengdu, China

  • Wang Jiaying,

    Affiliations Tianjin University of Traditional Chinese Medicine, Tianjin, China, Evidence Based Medicine Center in Tianjin, Tianjin, China

  • Hu Jing,

    Affiliation Peking University, Beijing, China

  • Xiong Jun

    Affiliation Department of Acupuncture, Jiang Xi Hospital of Traditional Chinese Medicine, Jiangxi Province, China

Abstract

Objective

To evaluate the reports' qualities which are about randomized controlled trials (RCTs) of acupuncture treatment on Mild Cognitive Impairment (MIC).

Methods

Nine databases including the Cochrane Central Register of Controlled Trials (CENTRAL,2010), PUBMED (1984-5/2010), EMbase (1984-5/2010), MEDLINE (1984-5/2010), CINAL (1984-5/2010), China National Knowledge Infrastructure (CNKI, 1980-5/2010), China Biomedicine Database disc (CBMdisc, 1980-5/2010), VIP (a full text issues database of China, 1989-5/2010) were searched systematically. Hand search for further references was conducted. Language was limited to Chinese and English. We identified 14 RCTs that used acupuncture as an intervention and assessed the quality of these reports against the Consolidated Standards for Reporting of Trials (CONSORT) statement and Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA).

Results

In regard to the items in the CONSORT statement, 13(92.86%) RCTs described baseline demographic and clinical characteristics in each group. 7 (50.0%) mentioned the method of generating the random sequence, only 2 (14.3%) RCTs had adequate allocation concealment. No RCTs used blinding. RCTs reported the sample size calculation. In regard to the items in STRICTA, 10 (71.43%) mentioned the depths of insertion, 6 (42.86%) reported acupuncture response, 11 (78.57%) mentioned the technique of acupuncture, 12 (85.71%) recorded the time, and only 3 (21.43%) RCTs reported the numbers of needles inserted. No RCTs reported the background of the acupuncture practitioners and professional title of practitioners.

Conclusion

The reporting quality of RCTs of acupuncture for mild cognitive impairment was moderate to low. The CONSORT statement and STRICTA should be used to standardize the reporting of RCTs of acupuncture in future.

Introduction

Mild Cognitive Impairment (mild cognitive disorder) is a clinical state with disease characteristics between normal aging and mild dementia. Most clinical researches show that 44% of the patients diagnosed as cognitive impairment turn into Alzheimer's transit (AD) within 3 years in follow-up studies at an annual average rate of 15% [1]. Recognizing cognitive impairment and identifying individuals at high risk of dementia have become a hot issue in dementia studies in recent years. Lacking in effective drugs and rehabilitation treatment, many patients with mild cognitive impairment begin to try alternative and complementary medicine therapies which include acupuncture treatment. However, despite the publication of a number of case-based research papers the effectiveness of acupuncture on mild cognitive impairment is to be tested and to do so a reliable scientific method is needed. Some research teams decide to adopt the randomized controlled trials (RCTs). RCTs, also known as randomized clinical trials, is a prospective study to test the efficacy and effectiveness of certain treatment or protective measures after comparing the results of the treatment group with those of the control group, thus the subjects involved in the study being randomized divided into two groups [2]. At present, randomized controlled trials (RCTs) is generally considered as the best design plan to verify the efficacy of intervention measures for it boasts rigorous design and scientific methods. It contributes to ensure high-quality systematic reviews, health technology assessment reports, and a variety of other reports for decision analysis, which meets the top requirements of evidence-based medical research [3]Therefore, we have adopted the internationally recognized standards for trial reporting (the CONSORT statement) [4] and the international standards for clinical trials of acupuncture interventions (STRICTA) [5] to evaluate the quality of reports of randomized controlled trials of acupuncture treatment for mild cognitive impairment in order to shed light on its further improvement.

Methods

Search strategy

The nine databases we have searched include Cochrane Central Register of Controlled Trials(CENTRAL,2010), PUBMED(1984-5/2010), EMbase(1984-5/2010), MEDLINE(1984-5/2010), CINAL(1984-5/2010), China National Knowledge Infrastructure(CNKI, 1980-5/2010), China Biomedicine Database disc(CBMdisc, 1980-5/2010), and VIP(a full text issues database of China,1989-5/2010). The work was done comprehensively and systematically and further references were searched by hand. Our search is featured by a wide coverage of reports and prospective studies in this field to provide enough information for research.

However, our search confined itself to reports written or published either in Chinese or in English. For one thing, it is because a majority of journal papers on acupuncture are published in these two languages and it is apparently impossible for reports written in Chinese to be included in non-Chinese databases. Besides, our team members were unable to read reports in languages other than Chinese and English. It may constitute to a limitation of this study.

We did not limit our search to a specific publication date range at the beginning of this study because we wanted to find as many reports as possible concerning mild cognitive impairment in the above databases. However, the databases themselves have such limitations.

Eligibility of the study

We included original reports in Chinese or English that described acupuncture as an intervention, stated the method, and provided specific data in this study and excluded commercials, speeches, letters and meeting reports.

Selection of reports to be studied

At first, one researcher (XL) picked out duplications of these reports using NoteExpress, and scanned the title and abstract of the citation retrieved by the selection search engine (first scanning). Another researcher (WJY) then viewed the full text of all potentially eligible reports obtained. After that, similar reports were marked “suspicious duplications” and compared. If the similarity rate of these articles was beyond 80% (including 80%), the one with the fullest information was included while the others were omitted. Disagreements between the two researchers were discussed by the whole team and were solved at last. (Figure 1)

thumbnail
Figure 1. Flow chart of reports selection.

This figure shows the process of the selection. The researchers applied the search method to find 359 reports related to the topic. One researcher (XL) picked out 19 duplications of these reports by the software NoteExpress and 169 reports of animals experiments, 5 cases and 36 reports of the wrong intervention by scanning the title and abstract of the citation retrieved by the selection search engine (first scanning). Another researcher (WJY) then viewed the full text of all potentially eligible reports obtained and picked out 73 reviews and theory analyses, one suspicious duplicate and 42 reports of the wrong intervention. At last, 14 reports are included for final analysis.

https://doi.org/10.1371/journal.pone.0016922.g001

Data abstraction

Two researchers (XL and WJY) compiled a table to list the general information of the final included reports. Items of the table include report title, journal name, year of publication, type of control, number of participants and country of publication. Any missed information was coded as “not reported” and disagreements were resolved after getting more information from related references before finalizing the table. Two reviewers scanned the full text of all 14 reports and filled the table.

Assessment methods

We assessed the quality of these reports by CONSORT and STRICTA standards. CONSORT (consolidated standards of reporting trails) was introduced in 1996 [6] in an attempt to alleviate the problems arising from inadequate reporting, and it was revised five years later [7]. STRICTA (standards for reporting interventions in controlled trials of acupuncture) was compiled between late 2001 and early 2002 [8].

The CONSORT statement comprises a 22-item checklist including abstract, background, participants, interventions, objectives, outcome measures, sample size, sequence generation, allocation concealment, implementation, blinding, statistical methods, participant flow, recruitment, demographic data, numbers analyzed, outcomes and estimation, ancillary analyses, adverse events, interpretation and overall evidence.

STRICTA consists of 15 items including acupuncture rationale, points used, number of needle inserted depths of insertion, response elicited, needle stimulation, needle retention time, needle type, treatment regimen, co-interventions, practitioner background, duration of training, length of clinical experience, expertise, control interventions. The above information can also be found in Table 2 and Table 3.

Two researchers were required to study the CONSORT and STRICTA standards before making assessment to ensure correct understanding of every item listed in the two checklists. If they had any difficulties or disagreements in the process, they solved the problem in discussions. The 14 reports were then assessed according to the two standards respectively and each item was marked “M” (mentioned) or “NM” (not mentioned).

Results

Report selection

A total of 359 potentially relevant reports were identified from the databases for review. 229 publications were considered to be unfit for our study after a preliminary review of title and abstract. Another 74 were excluded after full-text review. At last a total of 14 full-text reports [9][22] were regarded eligible for a more systematic analysis.

Characteristics of the Reports

The characteristics of all the included reports are presented in Table 1. All the 14 reports were found in 11 different medical journals published by six different publishing houses. Sample sizes ranged from a small pilot study of 17 subjects to a large scale, and the largest sample included 104 subjects. Only one report was published in America. All the 14 studies included in our assessment were published during the year 2006–2010, while in 2009 the number of reports published reached the peak (43%) (Figure 2).

thumbnail
Figure 2. Number of reports published in the recent five years.

According to the Table 1, the years of publication of included reports is mainly during these five years, namely from the 2006 to 2010. Figure 2 shows one report is published in 2006, two in 2007, four in 2008, six in 2009, and one in 2010.

https://doi.org/10.1371/journal.pone.0016922.g002

thumbnail
Table 1. Characteristics of the reports included (N = 14).

https://doi.org/10.1371/journal.pone.0016922.t001

Use of CONSORT and STRICTA

At first we found that none of the reports included in the assessment explicitly mentioned the CONSORT or STRICTA statement. After referring to the official websites of the 11 journals, we found that only two journals require manuscripts to be submitted to meet the CONSORT or STRICTA standards. However, we found out later that there were 3 reports that included 19 items of CONSORT and 12 items of STRICTA checklist, which means that the two statements have actually been employed by more authors.

Reporting Assessment Using CONSORT and STRICTA

The results are presented in Table 2 and Table 3. Not a single report fulfilled either all the 21 items of the CONSORT checklist or the 15 STRICTA items. Above all, the main problem we found was a lack of information concerning method of blinding, practitioner background, and participant flow and recruitment.

thumbnail
Table 3. Reporting quality of interventions in 14 RCTs based on STRICTA.

https://doi.org/10.1371/journal.pone.0016922.t003

CONSORT.

Seven CONSORT items were mentioned in less than 30% of the total reports. They are methods of blinding, participant flow and recruitment (0%), implementation of randomization (7.14%), description of allocation concealment (14.29%), adverse events of acupuncture (21.43%) and outcomes of ancillary analyses (28.57%).

Another four CONSORT items were included in more than half of all the reports, i.e., abstract, sequence generation and overall evidence in the section of discussion.

STRICTA.

None of these studies gave any information about practitioner background which was made up of expertise, duration of training and length of clinical experience. Two STRICTA items, namely, number of needles inserted and co-interventions were explained in 21.43% of the reports. Another two items were made clear in less than half of the reports, i.e., needle type (50%) and response elicited.

Discussion

Summary of the study

The main strength of our study is that we have searched a total of nine databases in order to encompass all the related reports. Moreover, reports selection, data collection, reports assessment and all the other work were carefully conducted by well-trained researchers.

Another important strength of our study is that we use the CONSORT and STRICTA statement to evaluate RCTs of acupuncture. Both of them are the most widely accepted criteria for reporting quality assessment in acupuncture studies. Plain in words and ready for use, the two statements were developed in the aim of alleviating the problems arising from inadequate reporting of randomized controlled trials. They are powerful tools, as has been proved by the fact that the majority of their items were well reported, with a few exceptions (7 CONSORT items and 9 STRICTA ones were poorly reported).

We believe it is high time to call for clinical researchers to follow the two statements in conducting RCTs and in thesis writing. At present, Chinese journals set down no requirements for the author to follow the CONSORT or STRICTA statement in reporting Therefore, we recommend that mainstream Chinese journals make the application of CONSORT or STRCITA a prerequisite, so that transparency and integrity of RCTs can be ensured. More importantly, for policy makers, it is urgent to let more clinical researchers know about the CONSORT and STRICTA statement.

Limitations

Although we have assessed the 14 reports comprehensively and systematically, there are still some limitations. First, the reports included in our study were published exclusively in years between 2006 and 2010 although we didn't make any limitations on the date of publication during sample selection. The fact that the selected reports cover a narrow time span may harm the value of our study. However, it is worth mentioning that the whole concept of mild cognitive impairment was not put forward until 1999 by American scholar Petersen. Apparently it took some time for the diagnostic criteria to be introduced to China and to be widely accepted by Chinese scholars. In addition, it probably cost more time for acupuncture, a complementary and alternative intervention, to be involved in the treatment. Recently, some research groups propose the need to test the efficacy of acupuncture treatment after three or four years' clinical practice in cognitive impairment patients, thus RCTs were conducted. Nevertheless, at the beginning we didn't realize it was impossible for us to find any target report before 1999 if sample selection was conducted with “mild cognitive impairment” as a search word, which means we have done something useless and our search strategy remains to be improved.

According to resources and databases accessible to us, the earliest RCTs were carried out in 2006. However, there might be earlier articles in the database that we didn't find for certain reasons, such as language competence. We limited our search to reports written in Chinese and English, but the English report turned out to be one in number. It seems that our quality review was confined to domestic reports. Results may vary if we were able to include more reports in other languages, but this requires our team members to attain a higher level of language skills, a target we are confident to reach in the future.

Another limitation lies in that we didn't follow the rule of blinding when assessing and allocating the reports. Researches who did the scanning were directly involved in the evaluation process. This is because we believe two researchers are enough to evaluate the whole 14 selected reports and allocation was unnecessary. However, although the two researchers had studied the same version of the checklist items beforehand, they came out with different opinions on certain issues. This shows that assessment bias still exists.

Conclusion

Our study suggests that the reporting quality of recently published reports of RCTs of acupuncture treatment for mild cognitive impairment remains to be improved. However, we have found only one published report had formally appealed for the adoption of CONSORT in future research. With more articles published on this subject, it is necessary to call for the wide application of the CONSORT and STRICTA standards to improve the quality of reporting, and this study should be pursued further with a greater number of reports on this subject assessed and analyzed.

Acknowledgments

Give our thanks to the authors of the cited articles timely reply our questions and SHANG Hongcai professor's for his careful guidance.

Author Contributions

Conceived and designed the experiments: WJY XL SHC. Performed the experiments: XL WJY. Analyzed the data: HJ XL WJY. Contributed reagents/materials/analysis tools: HJ XJ. Wrote the paper: WJY XL. Resolved the disagreement and bias: SHC HJ XJ.

References

  1. 1. Gmdman M, Petemen RC, Morris JC, ADSC Cooperative Study (1996) Rate of dementia of the Alzheimer type in subjects with mild cognitive impairment[J]. Neurology 46: A403.
  2. 2. WANG Ying, HAO Guangyue, GAO Bing (2006) introduction of revised edition of CONSORT statement–the reliable standard to improve qualities reporting evaluation [J]. Journal of Baotou medicine college 355–357.
  3. 3. ZHANG Xiaoli, LI Jing, ZHANG Mingming (2006) Assessing the Reporting Quality of Randomized Controlled Trials on Acupuncture for Acute Ischemic Stroke Using the CONSORT Statement and STRICTA [J]. Chinese Journal of Evidence Based Medicine 6(8): 586690. etc.
  4. 4. Altman DG, Schulz K, Moher D, et al. (2001) The Revised Statement CONSORT for Reporting Randomized Trials Explanation and Elaboration. Ann Intern Med 134(8): 663–694.
  5. 5. Mac Pherson H, White A, Cummings M, et al. (2002) STRICTA(Standards for Reporting Interventions in Controlled Trials of Acupunture). Acupunture in Medicine 20(1): 22–25.
  6. 6. Begg C, Cho M, Eastwood S, Horton R, Moher D, et al. (1996) Improving the quality of reporting of randomized controlled trials, the CONSORT statement JAMA 276(8): 637–639.
  7. 7. Moher D, Schula KF, Altman D, CONSORT Group (Consolidated Standards of Reporting Trials) (2001) The CONSORT statement: Revised recommendations for improving the quality of repors of parallel-group randomized trials JAMA 285(15): 1987–1991.
  8. 8. MacPherson H, White A, Cummings M, Jobst K, Rose K, et al. (2001) Standards for reporting invention in controlled trials of acupunture: The STRICTA recommendations. Complement Ther Med 8(1): 216–249, 20(1):22-25, 8(1):85-89.
  9. 9. He Xia (2008) Clinical Research on Mild Cognitive Impairment Treated by Combination of Acupunture and Medicine. JOURNAL OF PRACTICAL TRADITIONAL CHINESE MEDICINE ZYAO 2008(11).
  10. 10. Yu Tao (2007) Acupunture Treatment of Vascular Cognitive Impairment without Dementia in 31 Cases. SHAANXI JOURNAL OF TRADITIONAL CHINESE MEDICINE SXZY 2007(06).
  11. 11. Sun Yuanzheng, Zhang Ying (2009) Clinical Research on Mild Cognitive Impairment Treated by Combination of Acupunture and Duxil Tablet. JOURNAL OF CLINICAL ACUPUNTURE AND MOXIBUSTION ZJLC 2009(09).
  12. 12. Hou Xiaobing, et al. (2010) The Impact of Acupunture on Mild Cognitive Impairment in Functional Magnetic Resonance Imaging. WORLD JOURNAL OF INTEGRATED TRADITIONAL AND WESTERN MEDICINE SJZX 2010(02).
  13. 13. Hou Xiaobing (2009) Brain Functional Magnetic Resonance Imaging (fMRI) Changes in Leukoaraiosis with Mild Cognitive Impairment before and after Treatment of Acupunture. Beijing University of Traditional Chinese Medicine.
  14. 14. Sun Yuanzheng, Zhu Pengyu, Zhang Yan (2007) 30 Cases of of Mild Cognitive Impairment Treated with Acupunture and Oral Donepezil. Chinese Journal of Traditional Medical Science and Technology. TJYY. (03).
  15. 15. Su Zhiwei, Liu Huanrong (2006) 100 Cases of Mild Cognitive Impairment Treated with Yishen Tongdu Method Liaoning Journal of Traditional Chinese Medicine LNZY 2006(08).
  16. 16. Zhou Xiaoping, et al. (2008) Effect of the Acupunture Method of Regulating Mild and Smooth Meridian on Patients with Mild Cognitive Impairment after Cerebral infarction. Shanghai Journal of Acupunture and Moxibustion SHZJ 2008(05).
  17. 17. Zhou Xiaoping, et al. (2008) Effect of the Acupunture Method of Regulating Mild and Smooth Meridian on Patients with Mild Cognitive Impairment. GUANGXI JOURNAL OF TRADITIONAL CHINESE MEDICINE GXZY 2008(02).
  18. 18. Liu Jie, Liu Zhiyan (2009) Clinical Observation on Electro-acupunture Treatment for Mild Cognitive Impairment of Deficiency of Kidney Essence. Shanghai Journal of Acupunture and Moxibustion. SHZJ 2009(06).
  19. 19. Chen Shangjie, et al. (2008) Effects of Different Acupunture Methods on Neurological Deficit Score and Quality of Life of Patients with Mild Cognitive Impairment. Journal of Emergency in Traditional Chinese Medicine ZYJZ 2008(03).
  20. 20. Chen Shangjie, et al. (2009) Effects of Different Acupunture Methods on SMS and MBI of Patients with Mild Cognitive Impairment. Chinese Journal of Traditional Medical Science and Technology. TJYY 2009(01).
  21. 21. Feng Chunqing, et al. (2009) Effects observation of “Qi-reinforcing, Blood-regulating and Body resistance-strengthening” Acupunture Method for Treating Mild Cognitive Impairment. Lin Journal of Traditional Chinese Medicine. 2009. 29. (10).
  22. 22. P C, C H, L JG (2009) Effects of electroacupunture treatment on impaired cognition and quality of life in Taiwanese stroke patients. Journal of alternative and complementary medicine (New York, N.Y.) 2009(10).