Skip to main content

Quality of randomized controlled trials reporting in the treatment of melasma conducted in China

Abstract

Background

There is no existing report on the quality of randomized controlled trials (RCTs) of melasma treatment currently conducted in China. This study aims to assess the quality of RCT- reporting in the treatment of melasma conducted in China.

Methods

Several databases were searched from their inception through to August 2014. In order to rate the report quality, we scored 1 for the item if it was reported in CONSORT 2010 and 0 for the item if it was not definitely stated or was not clear. For overall quality score (OQS), 13 items were scored and calculated with a range of 0 to 13. Five items (‘randomization’, ‘allocation concealment’, ‘blinding’, ‘baseline characteristics’ and ‘intention-to-treat (ITT) analysis’) were scored and a key methodological index score (MIS) was calculated with a range of 0 to 5 for each trial.

Results

A total of 246 relevant RCTs were included in the final analysis. The median OQS was 7, with a minimum of 4 and a maximum of 11. Some items’ information was insufficient, especially in the categories of ‘trial design’, ‘sample size’, ‘recruitment’ and ‘ancillary analyses’ with a positive rate of less than 20%. The median MIS was 1 with a minimum of 0 and a maximum of 3. Some items’ reporting was poor, especially in the categories of ‘randomization’, ‘allocation concealment and implementation’, ‘blinding’ and ‘ITT analysis’ with a positive rate of less than 10%. The mean OQS increased by about 0.52 for manuscripts published in the period of 5-year increments (95% CI: 0.32 to 0.72; P < 0.001). With regard to the MIS, no variable was statistically significant in the ordinal regression model.

Conclusion

The reporting quality of RCTs in the treatment of melasma conducted in China is not satisfactory especially in key methodological items. Reporting of RCTs in this field should meet and keep up with the standards of the CONSORT statement.

Peer Review reports

Background

Melasma is an acquired increased pigmentation of the skin characterized by symmetrical and confluent grey-brown patches mostly on the areas of the face exposed to the sun, such as over the cheek bones, forehead, and chin [1]. Although the prevalence of melasma has not been investigated in most countries, melasma accounts for about 4 to 10% of new cases in dermatology hospitals [2] and is most common in Hispanics, darker-skinned patients, and Asians [3]. In China, the overall prevalence of melasma is 3.23 to 13.61%, being 0.36 to 8.33% for males and 4.65 to 17.98% for females [4,5].

The causes of melasma are still unclear and the treatment is often unsatisfactory. In general, available therapies consist of preparations applied to the skin (for example, sunscreens that block ultraviolet light, topical steroids, topical retinoids, azelaic acid and kojic acid) and, recently, laser therapy. In China, internal Chinese medicine, acupuncture, moxibustion and combined therapies are also common in the treatment of melasma. To find the more efficient therapies, more and more randomized controlled trials (RCTs) in melasma have recently emerged. However, it is still unknown what the reporting quality of RCTs is in melasma studies conducted in China. RCTs, as a ‘gold standard’ of evidence-based clinical practice, are generally considered to have the highest level of credibility in determining efficacy of a new treatment [6]. A well-conducted RCT reporting on melasma is an effective tool to scientifically present evidence for duplication, experts’ evaluations, peer review decisions and systematic reviews. Biased results of RCT reports with poor quality will mislead treatment decisions and the formulation of a national public health strategy [7,8].

The aim of this study was to assess the quality of RCT-reporting in the treatment of melasma conducted in China. Furthermore, we will be able to advise on improvements in the reporting quality in this field after we know the status.

Methods

Search strategy

The following databases were searched from their inception through to August 2014: the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, CINAHL, AMED, the Chinese Biomedical Database, China National Knowledge Infrastructure, Wanfang databases and VIP Information. Other resources were: reference lists of eligible studies and previous systematic reviews were also reviewed to identify further eligible studies. The following search terms were used in Chinese and English: melasma, melanosis, chloasma, randomized trials, RCT, China, and so on.

Inclusion and exclusion criteria

Types of studies: RCTs that assessed the effect of any intervention for melasma conducted in China. However, quasi-randomized trials, non-randomized, cross-over RCTs, case reports, and case-control were excluded. All RCTs had to be performed in China and mainly by Chinese researchers.

Types of interventions: we considered all types of interventions; this included studies that compared at least one active treatment with a control, which may be a placebo or an alternative intervention.

Types of participants: people of all age groups and ethnic backgrounds who had a clinical diagnosis of melasma made by a physician.

Assessment of reporting quality

Rating of overall reporting quality

In order to rate the report quality, we scored 1 for the item if it was reported in CONSORT 2010 [9,10] and 0 for the item if it was not definitely stated or was not clear. For overall quality score (OQS), 13 items were scored and calculated with a range of 0 to 13 (Table 1) [8,10,11].

Table 1 Overall quality of reporting rating using items from the Consolidated Standards for Reporting Trials (CONSORT) statement (n = 246)

Rating of key methodological items

Five key methodological categories of ‘randomization’, ‘allocation concealment’, ‘blinding’, ‘baseline characteristics’ and ‘intention-to-treat (ITT) analysis’ have been assessed separately because they relate to potential sources of bias [12-14]. We then developed eight ‘yes’/‘no’ items (Table 2), wording so that emphasis was placed on quality of reporting rather than adequacy of trial design. Each item was scored 1 if the method was appropriate and 0 if inappropriate or if the reporting was unclear. Five items were scored and a key methodological index score (MIS) were calculated with range of 0 to 5 for each trial.

Table 2 Reporting quality of key methodological items (n = 246)

Data extraction and analysis

Each article was reviewed by two independent investigators (YW and TY). Two investigators, blinded to each other’s ratings, completed the rating form independently. Statistics used a modified CONSORT checklist with 18 items (Tables 1 and 2). Cohen’s к-statistic was calculated to assess agreement between two assessors. Agreement was judged as poor if к ≤ 0.20; fair if 0.20 lower than к ≤ 0.40; moderate if 0.40 lower than к ≤ 0.60; substantial if 0.60 lower than к ≤ 0.80; good if к higher than 0.80; and perfect if к = 1 [8,10]. Discrepancies were reviewed in detail and subsequently settled by consensus.

To explore factors associated with OQS, a multiple linear regression model was performed only if variables in the univariate models were significant at P ≤ 0.10. In the final multivariate model, variables were significant predictors at P ≤ 0.05. As the OQS variable was in a normal distribution, we believed that employing a multiple linear regression model is enough to obtain an approximate P-value with the clarity of the coefficient interpretation (not the collinearity relationship between different independent variables from the result of the collinearity diagnosis) and the large number of cases (246 cases).

As the outcome variable, MIS could be considered as a count and the ordinal regression model was used to select factors associated with MIS. The independent factors for these models were ‘year of publication’, ‘funding’, ‘type of interventions’ and ‘type of journals’. Descriptive statistical analysis, linear and ordinal regression analysis were performed using SPSS version 20.0 (SPSS, Chicago, IL, USA). Analyses of Cohen’s к-statistics were performed using the SAS software, version 9.3 (SAS Institute, Inc., Cary, NC, USA). Database of RCTs reporting in the treatment of melasma conducted in China are provided in Additional file 1.

Results

The RCTs selection process is outlined in Figure 1. The researchers applied the search method to find 567 reports related to the topic. After carefully selecting, a total of 246 relevant RCTs were included in the final analysis (Figure 1).

Figure 1
figure 1

Flowchart of the article selection process.

Characteristics of included trials

The number of RCT reports on the treatment of melasma conducted in China is increasing. In the periods of ‘before 1999’, ‘2000 to 2004’, ‘2005 to 2009’ and ‘2010 to 2014’, 7 articles (2.8%), 13 articles (5.3%), 95 articles (38.6%) and 131 articles (53.3%), respectively emerged. For the journal type, 67 (27.2%), 45 (18.3%) and 134 (54.5%) were published in general medical journals, specialist medical journals and traditional and alternative medical journals, respectively. In terms of financial sources, 29 articles (11.8%) reported their funding sources, most of which were obtained from provincial and municipal sources. In terms of choice of interventions, the largest number of interventions were comprised of internal Chinese medicine (90 reports; 36.6%) followed by combined medicine (40 reports; 16.3%), drugs (25 reports; 10.2%), acupuncture and moxibustion (23 reports; 9.3%), ultrasonic therapy (6 reports; 2.4%), laser therapy (5 reports; 2.0%) and others (57 reports; 23.2%).

Quality of reporting

Rating of overall reporting quality

The overall quality of reporting rating using items from the CONSORT statement was listed in Table 1. The overall level of reporting was not good; the median OQS was 7, with a minimum of 4 and a maximum of 11. Some items’ information was insufficient, especially in the items of ‘trial design’, ‘sample size’, ‘recruitment’ and ‘ancillary analyses’ with a positive rate of less than 20%. A substantial agreement was observed for items 2, 3, 5, 7, 14, 18 and 19; a good agreement was observed for items 4, 6, 12, 13 and 17; a perfect agreement was observed for item 1.

Rating of key methodological items

The median MIS was 1 with a minimum of 0 and a maximum of 3. Some items’ reporting was poor, especially in the items of ‘randomization’, ‘allocation concealment and implementation’, ‘blinding’ and ‘ITT analysis’ with a positive rate of less than 10% (Table 2). A substantial agreement was observed for items 9, 10 and 11; a good agreement was observed for item 8, 15 and 16.

Exploratory analysis: factors associated with better reporting quality

In the univariate analysis, year of publication was the only factor associated with an increased OQS. In detail, the mean OQS increased by about 0.52 for manuscripts published in the period of 5-year increments (95% CI: 0.32 to 0.72; P < 0.001) (Table 3). With regard to the MIS, no variable was statistically significant in the ordinal regression model.

Table 3 Simple linear regression analysis for factors associated with better overall quality of reporting rating using items from the Consolidated Standards for Reporting Trials (CONSORT) statement (n = 246)

Discussion

Our study found that the reporting quality of overall and key methodological items was not satisfactory in RCTs of melasma conducted in China. This indicated that RCTs in this field still needed improvement to meet the standard of reporting quality required by the CONSORT statement. There seemed to be an improvement in time for OQS as the mean OQS increased by about 0.52 for manuscripts published in the period of 5-year increments, which indicated that more and more scholars were realizing the importance of reporting in RCTs due to the widely adoption and promotion of CONSORT.

However, we identified some items where information was insufficient or inadequate in most studies, such as ‘randomization’, ‘allocation concealment and implementation’, ‘blinding’, ‘ITT analysis’, and so on. These key methodological items are critical in avoiding selection, performance/detection, and attrition bias. An overestimation of treatment effects has been demonstrated in trials with inadequate key methodological design comparison with trials that adequately reported these methodological items [15]. Some previous studies also found it was easy to ignore the reporting importance of some key methodological items with a positive rate of less than 30% and similar overall quality of reporting had the possibility of hiding important differences in methodological quality [7,8,10,16]. Especially, for studies in RCTs of melasma mainly conducted outside China, it is reported that there is a positive rate of 65%, 20% and 95% for items of adequate ‘randomized sequence generation’, ‘allocation concealment’ and ‘blinding’, respectively [17]. This positive rate is higher than that in our study. Other studies [16,18,19] conducted in China in other fields reported that the categories of ‘randomization’, ‘blinding’ and ‘ITT’ were used in about 15 to 30%, 0 to 20% and 0 to 10% of trials, respectively, which were similar to the results in our study. Meanwhile, ‘trial design’, ‘sample size’, ‘recruitment’ and ‘ancillary analyses’ should be emphasized in reports.

One limitation we should point out is that, due to the language barrier, we did not search for any manuscripts published in non-Chinese or English journals. This is a worthwhile area for future study. It remains unknown whether searching in other language journals would have altered the constitution of our sample or results. Despite this limitation, we think our results have good internal validity. Firstly, it is a meaningful topic to assess the quality of RCT- reporting in the treatment of melasma conducted in China as more and more RCTs in this field are emerging and a full review is lacking. Secondly, in our survey, the selection and abstraction processes were independently performed by two qualified assessors who were well-trained in mastering the CONSORT statement and have good agreement in item evaluation reporting.

Conclusions

Our findings demonstrate that the reporting quality of RCTs in the treatment of melasma conducted in China is lacking especially in key methodological items. Reporting of RCTs in this field should meet and keep up with the standards of the CONSORT statement. It is believed that with the guidance of the CONSORT statement, more and more high quality RCT-reporting on the treatment of melasma conducted in China will emerge over time.

Abbreviations

CENTRAL:

Cochrane Central Register of Controlled Trials

CONSORT:

Consolidated Standards for Reporting Trials

ITT:

intention-to-treat

MIS:

methodological index score

OQS:

overall quality score

RCT:

randomized controlled trial

References

  1. Jutley GS, Rajaratnam R, Halpern J, Salim A, Emmett C. Systematic review of randomized controlled trials on interventions for melasma: an abridged Cochrane review. J Am Acad Dermatol. 2014;70(2):369–73.

    Article  PubMed  Google Scholar 

  2. Failmezger C. Incidence of skin disease in Cuzco. Peru Int J Dermatol. 1992;31(8):560–1.

    Article  CAS  PubMed  Google Scholar 

  3. Sanchez MR. Cutaneous diseases in Latinos. Dermatol Clin. 2003;21(4):689–97.

    Article  PubMed  Google Scholar 

  4. Wang R, Wang T, Cao L, Shen Y, Zhou C, Ding X, et al. Prevalence of melasma in Chinese Han and Chinese Yi: a survey in Liangshan district. Chin J Dermatovenereol. 2010;24(6):546–8.

    Google Scholar 

  5. Liu Q, Gao X, Liu X, Li H, Wang T, Wang X, et al. An epidemiological survey on chloasma in Zibo district, Shandong province. Chin J Dermatovenereol. 2013;27(3):283–5.

    Google Scholar 

  6. Li JY, Zhang YF, Smith GS, Xue CJ, Luo YN, Chen WH, et al. Quality of reporting of randomized clinical trials in tai chi interventions - a systematic review. Evid Based Complement Alternat Med. 2011;2011:383245.

    PubMed  PubMed Central  Google Scholar 

  7. Lu L, Luo G, Xiao F. A retrospective survey of the quality of reports and their correlates among randomized controlled trials of immunotherapy for Guillain-Barre syndrome. Immunotherapy. 2013;5(8):829–36.

    Article  CAS  PubMed  Google Scholar 

  8. Lai R, Chu R, Fraumeni M, Thabane L. Quality of randomized controlled trials reporting in the primary treatment of brain tumors. J Clin Oncol. 2006;24(7):1136–44.

    Article  PubMed  Google Scholar 

  9. Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Int J Surg. 2012;10(1):28–55.

    Article  PubMed  Google Scholar 

  10. Toulmonde M, Bellera C, Mathoulin-Pelissier S, Debled M, Bui B, Italiano A. Quality of randomized controlled trials reporting in the treatment of sarcomas. J Clin Oncol. 2011;29(9):1204–9.

    Article  PubMed  Google Scholar 

  11. Rios LP, Odueyungbo A, Moitri MO, Rahman MO, Thabane L. Quality of reporting of randomized controlled trials in general endocrinology literature. J Clin Endocrinol Metab. 2008;93(10):3810–6.

    Article  CAS  PubMed  Google Scholar 

  12. Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319(7211):670–4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Gluud LL, Thorlund K, Gluud C, Woods L, Harris R, Sterne JA. Correction: reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2008;149(3):219.

    Article  PubMed  Google Scholar 

  14. Montori VM, Bhandari M, Devereaux PJ, Manns BJ, Ghali WA, Guyatt GH. In the dark: the reporting of blinding status in randomized controlled trials. J Clin Epidemiol. 2002;55(8):787–90.

    Article  PubMed  Google Scholar 

  15. Chen B, Liu J, Zhang C, Li M. A retrospective survey of quality of reporting on randomized controlled trials of metformin for polycystic ovary syndrome. Trials. 2014;15:128.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Lu L, Liao M, Zeng J, He J. Quality of reporting and its correlates among randomized controlled trials on acupuncture for cancer pain: application of the CONSORT 2010 Statement and STRICTA. Expert Rev Anticancer Ther. 2013;13(4):489–98.

    Article  CAS  PubMed  Google Scholar 

  17. Rajaratnam R, Halpern J, Salim A, Emmett C, et al. Interventions for melasma. Cochrane Database Syst Rev. 2010;7:D3583.

    Google Scholar 

  18. Tang JL, Zhan SY, Ernst E. Review of randomised controlled trials of traditional Chinese medicine. BMJ. 1999;319(7203):160–1.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Lu L, Zeng J, Chen Y. Quality of reporting in randomized controlled trials conducted in China on the treatment of cancer pain. Expert Rev Anticancer Ther. 2011;11(6):871–7.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This research was supported by Chinese Medicine Administration of Zhejiang Province (Grant No. 2013ZA121). The funded project was entitled “Study of the effects of three flavonoids of semen astragali complanati on pigment of human epidermal cells co-culture system”. The funder had no further role in the study design, data collection, analysis and interpretation of data, writing of the report, or decision to submit the paper for publication.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yang Wang.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

ZWC, YCC and JCZ planned and designed the study. YW, TY and QCZ contributed to the acquisition of data. XJD, WTS and ZSD analyzed the data and YCC provided statistical support throughout the working process. ZWC, YCC and JCZ was primarily responsible for writing the paper. All authors were involved in the drafting and revision of the paper and approved the final version of the manuscript for submission. All authors read and approved the final manuscript.

Additional file

Additional file 1:

RCTs reporting in the treatment of melasma conducted in China.

Rights and permissions

Open Access  This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/.

The Creative Commons Public Domain Dedication waiver (https://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Z., Chen, Y., Zeng, J. et al. Quality of randomized controlled trials reporting in the treatment of melasma conducted in China. Trials 16, 156 (2015). https://0-doi-org.brum.beds.ac.uk/10.1186/s13063-015-0677-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13063-015-0677-2

Keywords