Figures
Abstract
Background
Bias in randomized controlled trials (RCTs) of complementary therapy interventions seems to be associated with specific factors and to potentially distort the studies’ conclusions. This systematic review assessed associated factors of risk of bias and consequences for the studies’ conclusions in RCTs of yoga as one of the most commonly used complementary therapies.
Methods
Medline/PubMed, Scopus, IndMED and the Cochrane Library were searched through February 2014 for yoga RCTs. Risk of selection bias was assessed using the Cochrane tool and regressed to a) publication year; b) country of origin; c) journal type; and d) impact factor using multiple logistic regression analysis. Likewise, the authors’ conclusions were regressed to risk of bias.
Results
A total of 312 RCTs were included. Impact factor ranged from 0.0 to 39.2 (median = 1.3); 60 RCT (19.2%) had a low risk of selection bias, and 252 (80.8%) had a high or unclear risk of selection bias. Only publication year and impact factor significantly predicted low risk of bias; RCTs published after 2001 (adjusted odds ratio (OR) = 12.6; 95% confidence interval (CI) = 1.7, 94.0; p<0.001) and those published in journals with impact factor (adjusted OR = 2.6; 95%CI = 1.4, 4.9; p = 0.004) were more likely to have low risk of bias. The authors’ conclusions were not associated with risk of bias.
Conclusions
Risk of selection bias was generally high in RCTs of yoga; although the situation has improved since the publication of the revised CONSORT statement 2001. Pre-CONSORT RCTs and those published in journals without impact factor should be handled with increased care; although risk of bias is unlikely to distort the RCTs’ conclusions.
Citation: Cramer H, Langhorst J, Dobos G, Lauche R (2015) Associated Factors and Consequences of Risk of Bias in Randomized Controlled Trials of Yoga: A Systematic Review. PLoS ONE 10(12): e0144125. https://doi.org/10.1371/journal.pone.0144125
Editor: Terence J. Quinn, University of Glasgow, UNITED KINGDOM
Received: September 15, 2015; Accepted: November 15, 2015; Published: December 2, 2015
Copyright: © 2015 Cramer et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All included data are available from the original articles included in this systematic review.
Funding: This review was supported by a grant from the Rut- and Klaus-Bahlsen-Foundation. The funding source had no influence on the design or conduct of the review; the collection, management, analysis, or interpretation of the data; or in the preparation, review, or approval of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Rooted in Indian philosophy and spiritual practice, the original goal of yoga has been described as quieting one’s mind to achieve the union of mind, body and spirit [1]. Regardless of its spiritual origins, yoga has become a popular route to physical and mental well-being [1] and has been adapted for use in complementary and alternative medicine in Western society [2]. In this setting, yoga is most often associated with physical postures, breath control and meditation; and different yoga schools have emerged that put varying focus on physical and mental practices [1].
Yoga is gaining increased popularity as a therapeutic practice; with more than 20 million Americans (9% of the USA's population) reporting they practiced yoga for health reasons in 2012 [3]. From 2002 to 2012, the prevalence of yoga use increased linearly, making it one of the most commonly used complementary and alternative medicine (CAM) approaches in the US [4]. This increased use is paralleled by an increasing amount of randomized controlled trials (RCTs) of yoga; with more than 50 RCTs being published each year now [5]. Those RCTs have shown benefits of yoga for various conditions including chronic pain [6,7], cancer-related symptoms [8,9], and depression [10,11]; as a preventive means [12]; and in educational settings [13].
The methodological quality of the available evidence on complementary therapies as a whole has however been frequently questioned [14,15]. Bias in RCTs on complementary therapy interventions has been shown to depend on the studies’ origin and journal type the study is published in; with RCTs that were conducted outside the USA [16], published in CAM specialty journals [15], and/or in journals without or with low impact factor [15] being more likely to be biased towards positive results. Specifically, recent systematic reviews on yoga were frequently unable to draw definite conclusions due to the high risk of bias of the included RCTs [17–19]. Comparably to complementary therapies as a whole, it has been shown that RCTs on yoga are far more likely to be positive when they were conducted in India [20]; potentially representing an influence of location on publication bias. It has however not yet been investigated what factors might influence risk of bias in individual studies on complementary therapy interventions and whether risk of bias in individual studies distorts the studies’ conclusions towards a positive or negative interpretation of the findings. Given that a detailed analysis of all RCTs on all complementary therapies would include several thousand RCTs on very diverse interventions, this systematic aimed to assess associated factors and consequences of risk of bias in RCTs of yoga as one of the most commonly used complementary therapies [2]. A decent knowledge of underlying factors of risk of bias would help judge the applicability of yoga-based RCTs without an explicit assessment of risk of bias. Knowing the influence of risk of bias on study conclusions would be helpful when judging the expressiveness of single studies e.g. for conducting systematic reviews or designing medical guidelines. Finally, if underlying factors of risk of bias are known, caveats against those factors can be applied when designing future yoga RCTs.
The aim of this systematic review thus was to evaluate a) what factors are associated with risk of bias; and b) whether risk of bias influences conclusions in RCTs of yoga. RCTs on clinical as well as on non-clinical populations were considered.
Methods
This systematic review is based on a previously published bibliometric analysis that descriptively summarized characteristics of RCTs of yoga [5]. Where applicable, reporting is in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines [21] (S1 PRISMA Checklist).
Eligibility criteria
Types of studies.
RCTs, cluster-randomized trials and randomized cross-over studies were eligible. No language restrictions were applied; if necessary, language experts were consulted. Duplicate publications; this is multiple articles reporting identical or different results on already published studies; were treated as a single study.
Types of participants.
Studies of all types of participants were eligible. No restrictions were made regarding sociodemographic characteristics or health status.
Types of interventions.
Studies were eligible if they compared yoga interventions to one or more non-yoga interventions or untreated control groups. No restrictions were applied regarding the tradition, length, frequency or duration of the studied yoga programs. The specific yoga practices included in the intervention were not restricted as long as the intervention was based on yoga theory and/or traditional yoga practices. Studies allowing individual co-interventions were eligible while studies on multimodal interventions that include yoga amongst others were excluded. Head-to-head comparisons of different yoga interventions without a non-yoga control group were also excluded.
Literature search methods
Four electronic databases were searched from their inception through February 12, 2014: Medline/PubMed, Scopus, IndMED and the Cochrane Library. The literature search was constructed around search terms for “yoga” and a filter for retrieving randomized controlled trials [5]. The complete search strategy for Medline/PubMed is shown in Table 1. The reference lists of identified original articles or reviews and the tables of contents of the Journal of Yoga & Physical Therapy and the International Scientific Yoga Journal SENSE, which are not listed in electronic databases, were searched manually for additional eligible studies. Identified abstracts were screened independently by two review authors; and potentially eligible articles were then read in full by two review authors to determine whether they actually met the eligibility criteria.
Data extraction
Bibliometric data (publication year, country of origin, journal of publication) were extracted independently from the included studies by two authors using a standardized data extraction form. The journals’ 2014 impact factor was extracted from Thompson Reuters’ Journal Citation Reports [22].
The conclusions by the authors of the original articles were extracted from the studies’ abstracts and rated by a reviewer blinded to the review’s aims as a) positive: the yoga intervention was stated to be helpful for a respective condition or symptom and/or to be superior to at least one non-yoga control group; b) neutral: no clear statement regarding helpfulness or superiority regarding a respective condition or symptom was made; or c) negative: the yoga intervention was stated to be unhelpful for a respective condition or symptom and/or to be inferior or not superior to all non-yoga control groups. Where conclusions were rated to be neutral or where no conclusion was provided in the study’s abstract, conclusions from the respective study’s discussion section were additionally checked.
Assessment of risk of bias in individual studies
Risk of bias was independently assessed by two reviewers using the Cochrane risk of bias tool [23]. Given that selection bias has been empirically shown to be the most important source of bias in RCTs [24], it was assessed as a marker of the studies’ overall risk of bias. Random sequence generation and allocation concealment were assessed as low; unclear; or high risk of bias. A low risk of bias regarding random sequence generation is assumed if an adequate random component would be included; such as random number tables, coin-tossing, or throwing a dice [23]. A high risk of bias is assumed if systematic, not strictly random, methods of allocation were used; including alteration, or allocation based on date of birth or case record number [23]. An unclear risk of bias is assumed if the information on random sequence generation were insufficient to judge the adequacy of the applied method; e.g. by simply describing the study as “randomized” without further information on the applied methods [23]. Regarding allocation concealment, a low risk of bias is assumed if proper methods were used to implement the random sequence to prevent foreknowledge of intervention assignments, including central randomization by a third party or the use of sequentially numbered, opaque, sealed envelopes which are opened sequentially [23]. A high risk of bias Is assumed if the sequence was openly accessible by the study staff [23]. An unclear risk of bias is assumed if the information on methods of allocation concealment was insufficient to judge its adequacy [23]. Discrepancies between reviewers were discussed with a third reviewer until consensus was reached.
Statistical analysis
Data were analyzed using IBM SPSS® Statistics for Windows (release 22.0. Armonk, NY: IBM Corp). Independent predictors of low risk of bias for a) random sequence generation; b) allocation concealment; and c) both domains together were identified using multiple logistic regression analysis. A backward stepwise procedure with a Wald statistic p-value of ≤0.05 with the following independent variables was used: a) publication year; b) country of origin; c) journal type of publication; and d) impact factor. Country of origin, i.e. the country where the trial was conducted, was categorized by continents as a) North America; b) Europe; c) Asia; and d) other continents. Journal of publication was categorized as a) CAM specialty journal; and b) other types of journals. Journals were categorizes as CAM specialty journals if they explicitly focused on complementary, alternative and/or integrative medicine in their aims and scopes. Journals explicitly focusing on yoga were also coded as CAM specialty journals. In a second analysis, binary predictors were used for publication year and impact factor. Publication year was categorized as a) being published before the publication of the revised CONSORT Statement 2001 [25], i.e. before 2002; and b) being published after the publication of the revised CONSORT Statement 2001; i.e. 2002 or later. Impact factor was categorized as a) journals without impact factor; and b) journals with impact factor. The second analysis was used to calculate adjusted odds ratios (OR) with 95% confidence intervals (CI) for each significant predictor. Finally, the influence of risk of bias on the articles’ conclusions was analyzed using impact factor as an independent variable and conclusion as the dependent variable. Conclusions were categorized as a) positive; or b) not positive. Given that it has previously been shown that Indian yoga RCTs are significantly more likely to reach positive conclusions than those from other countries [20], country of origin (a) India; b) other countries) was included in the analysis as a potential confounder.
Results
Study characteristics
A total of 312 RCTs were included (Fig 1). The RCTs were published between 1975 and 2014; 45 RCTs (14.4%) were published before 2001, and 267 RCTs (85.6%) in 2002 or later. Ninety-four RCTs (30.1%) originated from North-America, 37 (11.9%) from Europe, 161 (51.6%) from Asia, and 20 (6.4%) from other continents. Regarding journals, 86 RCTs (27.6%) were published in CAM specialty journals, and 226 (27.6%) in other types of journals. The journals’ impact factor ranged from 0.0 to 39.2; with a median of 1.3 (interquartile range 0.0 to 2.4); and with 131 RCTs (42.0%) being published in journals without impact factor, and 181 (58.0%) in journals with impact factor. The authors’ conclusions were positive in 278 RCTs (89.1%), neutral in 19 RCTs (6.1%), and negative in 15 RCTs (4.8%).
Risk of bias in individual studies
Regarding random sequence generation, 120 RCTs (38.5%) had low risk of bias; 185 RCTs (59.3%) had unclear risk of bias; and 7 (2.2%) had high risk of bias. A total of 70 RCTs (22.4%) had low risk of bias; 234 RCTs (75.0%) had unclear risk of bias; and 8 (2.6%) had high risk of bias regarding allocation concealment. Combining both sources of selection bias, 60 RCT (19.2%) had a low risk of selection bias, and 252 (80.8%) had a high or unclear risk of selection bias.
Regression analyses
Using linear predictors, only publication year and the journal’s impact factor significantly predicted low risk of bias regarding random sequence generation (p<0.001 and p = 0.015, respectively), allocation concealment (p<0.001 and p = 0.002, respectively), or both (p<0.001 and p = 0.003, respectively; Figs 2 and 3). RCTs published after the publication of the revised CONSORT Statement 2001 had 12.6 times the odds of having low risk of selection bias of those published earlier; and those published in journals with impact factor had 2.6 times the odds of those published in non-impact factor journals (Table 2). In fact, only 1 out of 45 pre-CONSORT RCTs (2.2%) had low risk of selection bias compared to 59 out of 267 post-CONSORT RCTs (22.1%; Fig 2); and 15 out of 131 RCTs from non-impact factor journals (11.5%) compared to 45 out of 181 RCTs from journals with impact factor (24.9%; Fig 3).
B, regression coefficient; CI, confidence interval; OR, odds ratio; SE, standard error.
Controlling for the RCT’s origin, low risk of bias regarding random sequence generation (p = 0.176), allocation concealment (p = 0.694), or both (p = 0.663) did not predict the direction of the authors’ conclusions; 225 out of 252 RCTs with unclear or high risk of selection bias (89.3%) reached positive conclusions compared to 53 out of 60 RCTs with low risk of selection bias (88.3%; Table 3).
Discussion
Summary of evidence
In this systematic review of 312 RCTs of yoga, only about 19% of the included trials had low risk of selection bias; and risk of bias was strongly associated with publication year. Specifically, although still highly prevalent, the percentage of trials with high or unclear risk of bias dramatically decreased after the publication of the revised CONSORT statement 2001. Further, risk of bias was negatively associated with the impact factor of the journal the respective RCT was published in; but not with country of origin and journal type. About 89% of the RCTs reached positive conclusions, however conclusions were not distorted by risk of bias.
Several findings deserve attention. Firstly the number of trials with low risk of selection bias significantly increased over time. Such increases have been found before in general medicine [25], palliative care [26] or in general medical journals as well as specialty journals [27]. The development and dissemination of reporting guidelines may be considered the main cause for these improvements of reporting of trials; with CONSORT [25,28] leading the way. However poor reporting remains common as this analysis found that only 19% of yoga trials reported adequate randomization and allocation concealment. High proportions of unclear or high risk of selection bias however are not just a problem of yoga research; they have been reported in other fields as well, for example in complementary and alternative medicine in general [14] nursing research [29], orthopedics [30] and in pediatrics [31]. Even studies published in high impact factor general medicine journals failed to reported adequate sequence generation in up to 40% of trials; and allocation concealment in up to 80% of trials from 2002 to 2003 [29]; unfortunately no recent data for those journals are available to compare the success rate in adhering to CONSORT.
Furthermore low risk of bias was associated with an existing impact factor. Previous studies have found associations between impact factor and risk of bias, with larger proportions of low risk trials being published in higher impact factor journals [32]. Of course these findings might be explained by different factors, for example trials published in higher impact factor journals differ from trials in other journals in many ways; they are more often funded, of larger size and they apply more elaborate statistics [32]. Such factors might be associated with better resources to adhere to high scientific standards of conduct and reporting. Furthermore higher impact factor journals eventually endorse the use of reporting guidelines, which in turn improves the reporting of relevant items significantly [33], even though it does not guarantee complete adherence to the guidelines [33,34]. In 2006 only 10% of journals stated that the submission of the CONSORT checklist as prerequisite for submission of manuscripts [35], and one can assume that those are probably the higher impact factor journals.
Other factors that might be taken into account when considering impact factors is the peer review process. A study revealed that one third of reviewers had not received formal training before conducting peer review of trials, even though more than 60% of them stated that the risk of bias assessment was one of their priorities when checking manuscripts [36]. Reviewers for higher impact factor journals may have more experience regarding the peer review, and the journals may even be able to provide checklist for the peer review process which are suggestive of improvement in reporting [37]. Furthermore high impact journals are often general medicine journals, and thus they may have access to a larger number of eligible reviewers. Specialty journals in general and CAM specialty journals in particular may be short of reviewers, especially reviewers with research experience and methodological training. Such journals could however provide checklists for reviewers and increase the quality of reporting of trials.
Last but not least yoga research may be special in several ways as the common guidelines may not be comprehensive enough. Studies have determined certain components of yoga interventions that should be reported in more detail [38–40]. Similar to the STRICTA statement for randomized controlled trials in acupuncture [41], a yoga addendum to the CONSORT statement may be useful to disseminate the use of reporting guidelines amongst yoga researchers.
Last but not least, results indicated that the risk of bias was not associated with the studies’ conclusions. This is in contrast with findings of associations of methodological quality of CAM trials and the likelihood of positive conclusions at least for trials published in CAM specialty journals without impact factor [15]. However, this analysis assessed study quality mainly by brief instruments such as the Jadad score rather than by more comprehensive multidimensional risk of bias tools [42]; and yoga was not assessed exclusively but only in conjunction with a variety of other CAM interventions. Thus, the present findings may indicate that other factors may be more predictive of conclusions in yoga trials than risk of selection bias. It has for example been demonstrated that Indian yoga RCTs had 25 times the odds of reaching positive conclusions than those conducted elsewhere [20]. This might be reflective of increased publication bias, inadequate statistical analysis or interpretation of findings in India; it might however also simply be due to a higher effectiveness of this indigenous practice in its country of provenance [20]. The latter interpretation would also be in line with the present findings of similar risk of bias in RCTs from different countries of origin; indicating comparable methodological quality across continents. While it has also been claimed that CAM specialty journals are more prone to positive publication bias than mainstream journals [43,44], this has not proven true for yoga RCTs whose likelihood of reaching positive results did not depend on journal type [20].
Methodological impact
The reporting of randomized controlled trials of yoga needs to be improved. Different approaches are possible and should receive attention. Journals should ensure that they endorse the use of reporting guidelines and provide links to relevant material, publications and training resources. They might also provide checklists for reviewer, or develop templates for authors and peer reviewers enforcing a comprehensive report. Last but not least the development and dissemination of a yoga-specific addendum to CONSORT might receive special attention within the research community and foster the dissemination of CONSORT and related guidelines.
Limitations
This analysis has a number of limitations. First, the definitions of yoga are manifold and the interventions included in this analysis ranged from highly demanding exercise-based interventions to more meditative low-impact interventions [5]. However, this is a general problem in yoga research and the exclusion of specific types of yoga might have biased the analysis toward specific time periods, countries of origin, or even single research groups. Second, no attempt was made to include unpublished trials, since the aim of this analysis included to assess the journal type, impact factor, and publication year of each included RCT. Third, only one dimension of risk of bias was assessed. As selection bias has been empirically shown to be the most important source of bias in RCTs [24], this approach seemed reasonable but might still have reduced the applicability of the findings to other sources of bias.
Conclusion
Although the situation has improved since the publication of the revised CONSORT statement in 2001, risk of selection bias was still high in RCTs of yoga. Pre-CONSORT RCTs and those published in journals without impact factor should be handled carefully when evaluating the helpfulness of yoga for a specific patient group. However, as risk of bias is unlikely to distort the trials’ conclusions, these RCTs can still be considered if no other sources of information are available.
Acknowledgments
This review was supported by a grant from the Rut- and Klaus-Bahlsen-Foundation. The funding source had no influence on the design or conduct of the review; the collection, management, analysis, or interpretation of the data; or in the preparation, review, or approval of the manuscript. The authors would like to thank Jana Hochstein, Essen, Germany, for her assistance in data extraction; and Dr. Petra Klose, Essen, Germany, and Dr. Hoda Azizi, Mashhad, Iran, for their assistance in assessing the Chinese, Japanese, and Farsi manuscripts.
Author Contributions
Conceived and designed the experiments: HC RL. Performed the experiments: HC RL. Analyzed the data: HC RL. Contributed reagents/materials/analysis tools: JL GD. Wrote the paper: HC RL. Approved the final manuscript: HC JL GD RL.
References
- 1.
Feuerstein G (1998) The yoga tradition. Prescott: Hohm Press.
- 2.
De Michelis E (2005) A History of modern yoga: Patanjali and Western esotericism. London, UK: Continuum International Publishing Group.
- 3. Cramer H, Ward L, Steel A, Lauche R, Dobos G, Zhang Y (2015) Prevalence, patterns, and predictors of yoga use: Results of a US nationally representative survey. Am J Prev Med.
- 4. Clarke TC, Black LI, Stussman BJ, Barnes PM, Nahin RL (2015) Trends in the use of complementary health approaches among adults: United States, 2002–2012. Natl Health Stat Report: 1–16.
- 5. Cramer H, Lauche R, Dobos G (2014) Characteristics of randomized controlled trials of yoga: a bibliometric analysis. BMC Complement Altern Med 14: 328. pmid:25183419
- 6. Büssing A, Ostermann T, Lüdtke R, Michalsen A (2015) Effects of yoga interventions on pain and pain-associated disability: a metaanalysis. J Pain 13: 1–9.
- 7. Cramer H, Lauche R, Langhorst J, Dobos G (2013) A systematic review and meta-analysis of yoga for low back pain. Clin J Pain 29: 450–460. pmid:23246998
- 8. Cramer H, Lange S, Klose P, Paul A, Dobos G (2012) Yoga for breast cancer patients and survivors: a systematic review and meta-analysis. BMC cancer 12: 412. pmid:22988934
- 9. Buffart LM, van Uffelen JG, Riphagen II, Brug J, van Mechelen W, Brown WJ, et al. (2012) Physical and psychosocial benefits of yoga in cancer patients and survivors, a systematic review and meta-analysis of randomized controlled trials. BMC Cancer 12: 559. pmid:23181734
- 10. Cramer H, Lauche R, Langhorst J, Dobos G (2013) Yoga for depression: a systematic review and meta-analysis. Depress Anxiety 20: 1068–83.
- 11. Gong H, Ni C, Shen X, Wu T, Jiang C (2015) Yoga for prenatal depression: a systematic review and meta-analysis. BMC Psychiatry 15: 14. pmid:25652267
- 12. Cramer H, Lauche R, Haller H, Steckhan N, Michalsen A, Dobos G (2014) Effects of yoga on cardiovascular disease risk factors: a systematic review and meta-analysis. Int J Cardiol 173: 170–83. pmid:24636547
- 13. Serwacki ML, Cook-Cottone C (2012) Yoga in the schools: a systematic review of the literature. Int J Yoga Therap (22): 101–9. pmid:23070680
- 14. Wieland LS, Manheimer E, Sampson M, Barnabas JP, Bouter LM, Cho K, et al. (2013) Bibliometric and content analysis of the Cochrane Complementary Medicine Field specialized register of controlled trials. Syst Rev 2: 51. pmid:23826877
- 15. Pittler MH, Abbot NC, Harkness EF, Ernst E (2000) Location bias in controlled clinical trials of complementary/alternative therapies. J Clin Epidemiol 53: 485–489. pmid:10812320
- 16. Sood A, Knudsen K, Sood R, Wahner-Roedler DL, Barnes SA, Bardia A, et al. (2007) Publication bias for CAM trials in the highest impact factor medicine journals is partly due to geographical bias. J Clin Epidemiol 60: 1123–1126. pmid:17938053
- 17. Cramer H, Lauche R, Klose P, Langhorst J, Dobos G (2013) Yoga for schizophrenia: a systematic review and meta-analysis. BMC Psychiatry 13: 32. pmid:23327116
- 18. Cramer H, Lauche R, Azizi H, Dobos G, Langhorst J (2014) Yoga for multiple sclerosis: a systematic review and meta-analysis. PLoS One 9: e112414. pmid:25390344
- 19. Posadzki P, Cramer H, Kuzdzal A, Lee MS, Ernst E (2014) Yoga for hypertension: a systematic review of randomized clinical trials. Complement Ther Med 22: 511–522. pmid:24906591
- 20. Cramer H, Lauche R, Langhorst J, Dobos G (2015) Are Indian yoga trials more likely to be positive than those from other countries? A systematic review of randomized controlled trials. Contemp Clin Trials 41: 269–272. pmid:25705015
- 21. Moher D, Liberati A, Tetzlaff J, Altman DG (2010) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 8: 336–341. pmid:20171303
- 22.
ISI Web of Knowledge (2015) Journal Citation Report.
- 23.
Higgins JPT, Green S (2010) Cochrane Handbook for Systematic Reviews of Interventions. West Sussex, UK: Wiley.
- 24. Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273: 408–412. pmid:7823387
- 25. Moher D, Jones A, Lepage L (2001) Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA 285: 1992–1995. pmid:11308436
- 26. Piggott M, McGee H, Feuer D (2004) Has CONSORT improved the reporting of randomized controlled trials in the palliative care literature? A systematic review. Palliat Med 18: 32–38. pmid:14982205
- 27. To MJ, Jones J, Emara M, Jadad AR (2013) Are reports of randomized controlled trials improving over time? A systematic review of 284 articles published in high-impact general and specialized medical journals. PLoS One 8: e84779. pmid:24391973
- 28. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. (1996) Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 276: 637–639. pmid:8773637
- 29. Smith BA, Lee HJ, Lee JH, Choi M, Jones DE, Bausell RB, et al. (2008) Quality of reporting randomized controlled trials (RCTs) in the nursing literature: application of the consolidated standards of reporting trials (CONSORT). Nurs Outlook 56: 31–37. pmid:18237622
- 30. Chess LE, Gagnier J (2013) Risk of bias of randomized controlled trials published in orthopaedic journals. BMC Med Res Methodol 13: 76. pmid:23758875
- 31. Jacobson DA, Bhanot K, Yarascavitch B, Chuback J, Rosenbloom E, Bhandari M (2015) Levels of evidence: a comparison between top medical journals and general pediatric journals. BMC Pediatr 15: 3. pmid:25879729
- 32. Bala MM, Akl EA, Sun X, Bassler D, Mertz D, Mejza F, et al. (2013) Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis. J Clin Epidemiol 66: 286–295. pmid:23347852
- 33. Turner L, Shamseer L, Altman DG, Schulz KF, Moher D (2012) Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev 1: 60. pmid:23194585
- 34. Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C, et al. (2006) Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust 185: 263–267. pmid:16948622
- 35. Hopewell S, Altman DG, Moher D, Schulz KF (2008) Endorsement of the CONSORT Statement by high impact factor medical journals: a survey of journal editors and journal 'Instructions to Authors'. Trials 9: 20. pmid:18423021
- 36. Chauvin A, Ravaud P, Baron G, Barnes C, Boutron I (2015) The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors. BMC Med 13: 158. pmid:26141137
- 37. Hirst A, Altman DG (2012) Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals. PLoS One 7: e35621. pmid:22558178
- 38. Ward L, Stebbings S, Sherman KJ, Cherkin D, Baxter GD (2014) Establishing key components of yoga interventions for musculoskeletal conditions: a Delphi survey. BMC Complement Altern Med 14: 196. pmid:24942270
- 39. Ward L, Stebbings S, Cherkin D, Baxter GD (2014) Components and reporting of yoga interventions for musculoskeletal conditions: a systematic review of randomised controlled trials. Complement Ther Med 22: 909–919. pmid:25440383
- 40. Sherman KJ (2012) Guidelines for developing yoga interventions for randomized trials. Evid Based Complement Alternat Med 2012: 143271. pmid:23082079
- 41. MacPherson H, Altman DG, Hammerschlag R, Youping L, Taixiang W, White A, et al. (2010) Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): Extending the CONSORT statement. J Evid Based Med 3: 140–155. pmid:21349059
- 42. Berger VW, Alperson SY (2009) A general framework for the evaluation of clinical trial quality. Rev Recent Clin Trials 4: 79–88. pmid:19463104
- 43. Ernst E, Pittler MH (1997) Alternative therapy bias. Nature 385: 480.
- 44. Schmidt K, Pittler MH, Ernst E (2001) Bias in alternative medicine is still rife but is diminishing. BMJ 323: 1071.