Introduction

Animals are often used during biomedical research studies, but such studies are often controversial because there are many differences between animals and humans. The aims of preclinical animal experiments are to perform preliminary safety and efficacy validations of the (new) intervention under study, with the results determining whether the intervention should be further assessed in clinical studies and trials. Every year, many animal experiments, including those supported by foundations and other organizations, are published in a variety of journals. In China, the National Natural Science Foundation of China (NSFC) is one of the major funding sources for basic science research.

Although random allocation and blinding are common in clinical trials, they are not common in animal experiments that precede clinical trials. One study[1] assessed 290 published animal experiments and reported that 32.41% (94/290) used randomization and 9.66% (28/290) used blinding. Non-randomized, non-blinded animal studies are more likely to report a difference between study groups than animal studies that use these methods[1]. Experimental design, statistical analysis, and reporting issues have also been found for animal studies [2,3].

The Animals in Research: Reporting In Vivo Experiments (ARRIVE) guidelines published in 2010 were developed to improve the transparency and accuracy of bioscience research reporting. The guidelines detail the minimum information that should be reported when using animals in a research study and include a 20-item checklist[4]. In 2011, the ARRIVE guidelines were introduced in China[5]. To date, no study has assessed whether animal experiments of neoplasms published in Chinese journals adhere to these guidelines.

The aim of this study was to determine the reporting quality of experiments, specifically those involving neoplasms in animals, which were published in Chinese journals between 2010 and 2012 and were supported by the NSFC.

Methods

Results

The literature search

We initially identified 2846 studies. Of these, 1889 were excluded given their title or abstract, and a further 520 were excluded after an assessment of their full texts because they did not meet the inclusion criteria. The full texts of an additional 41 articles were not available. The remaining 396 studies met the inclusion criteria and were included in the analysis (Fig 1 and S2 File).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Flow chart of articles identified, included and exclude.

https://doi.org/10.1371/journal.pone.0154657.g001

General Characteristics of the Included Studies

The characteristics of the included studies are shown in Table 1. Of these studies, 126 (31.82%), 141 (35.61%), and 129 (32.58%) were published in 2010, 2011, and 2012, respectively. There were 77 studies which were published the release of the before ARRIVE guideline. For slightly more than half of the studies (219/396; 55.30%), the first author was affiliated with a hospital. The most commonly studied conditions were liver and lung cancers. The majority of the studies randomly assigned animals to the experimental or control group, but only 26.77% (106/396) described the randomization method in detail. Eleven studies used a random number table to perform randomization. Other randomization methods are described in Table 2.A total of 159 studies (40.15%) were supported by only the NSFC. The other 237 studies (59.85%) were supported by up to five sources.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of included studies.

https://doi.org/10.1371/journal.pone.0154657.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Reporting of checklists for ARRIVE Guidelines.

https://doi.org/10.1371/journal.pone.0154657.t002

Discussion

Over the last decade, numerous studies have examined the quality of healthcare reporting by assessing the compliance of randomized controlled trials[6],observational studies[7], and systematic reviews[8] with various assessment instruments. The CONSORT statement published in 1996[9], the ARRIVE guidelines[4], and the Gold Standard Publication Checklist (GSPC)[10] published in 2010 represent substantial improvements in methods used for animal studies. Before these guidelines, insufficient reporting occurred in many animal experiments, e.g., a lack of randomization and/or blinding[11–13]. The reporting quality of animal experiments in periodontology, e.g., implant dentistry, published after 2010 has been assessed using the ARRIVE and modified ARRIVE guidelines[2,3,14–17].

In the present study, we focused on 396 reports involving neoplasms in rodents published in Chinese journals between 2010 and 2012. To our knowledge, this is the first assessment of this type of study published in Chinese journals and supported by the NSFC. Although we did not perform an intervention systematic review, we tried to conform to the preferred reporting items for the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA statement) checklist[8]. In our study, we found that the completeness of these studies regarding the ARRIVE guidelines was suboptimal. The mean ARRIVE checklist score was only 19.48 out of 40, and many studies failed to report important information. Furthermore, half the items in the ARRIVE checklist were reported by <50% of studies. Items 3b, 5, 6d, 6f, 7c, 7d, 9b, 9c, 10b, 10c, 11a, 13b, 13c, 14, 15a, 15b, 17a, 17b, 18b, 18c, and 19 were particularly poorly reported.

All studies included in this analysis were supported by at least one funding organization. Only two studies reported that no conflict of interest existed, and the other studies did not provide a declaration of interests. Accurate reporting of conflicts of interest is very important to enable readers to judge the risk of publication bias.

We found that few studies (1.77%) provided a statement of ethics or provided information on animal welfare. Researchers need to pay close attention to study design, data collection, reporting, and the welfare of the animals to take effective measures to alleviate animal suffering. The ARRIVE guidelines recommend providing a time-line diagram or flow chart of the study design, but neither was part of any study. Similarly, no study described how the sample size was calculated, and therefore it was unclear if the sample sizes were adequate. To minimize the effect of random error on the results, studies should include independent repetitions of each experiment. Although many studies reported the number of repeated measurements, many failed to describe the number of independent replications for each experiment.

As with clinical trials, animal experiments should report if the subjects were randomized into groups and also describe the randomization method in detail to help readers assess the risk of selective bias. Furthermore, it is important to blind outcome assessors and data analysts to the group assignment, thereby reducing the risk of measurement bias for some subjective outcomes. However, blinding was used in only one study.

Although all studies reported the experimental outcomes, many did not define primary and secondary outcomes. Information concerning statistical methods was often incomplete, e.g., the unit of analysis for each dataset was not specified, or the rationale for selecting the statistical approach was not provided.

Only one studies (0.25%) provided the relevant characteristics and health status of the animals, e.g., the strain, sex, age, and weight of the experimental and control animals, which are important factors that allow the reader to judge if the groups were or were not balanced at baseline.

Insufficient information was often provided on the number of animals studied, and no descriptions or explanations were provided concerning animal death or loss to follow-up or incomplete outcome data, making it difficult for readers to evaluate the risk of attrition bias. In addition, although all studies reported the efficacy of the treatment, only 41 (10.35%) reported adverse events, and none described the approach(s) used to reduce adverse events.

Ideally, the discussion section of such studies should address how the results translate to other systems or species and the feasibility of testing the treatment in clinical trials. When animals are used in research, the researcher has the responsibility to work toward the replacement, refinement, and reduction of animal use, i.e., to improve the use value of the animals, reduce the number of animals used, and increase the use of alternative methods. Unfortunately, none of the studies reported information about these issues.

All experimental research has limitations; however, many of the studies did not address this point.

Analysis reveals that the reporting quality of animal experiments reports shows a rising tendency after the first publication of the ARRIVE in June, 2010.However, the quality of reporting was still poor, and future studies should be more transparent and accurately reported. We believe that reporting guidelines, including those of ARRIVE and GSPC, have not been widely used by Chinese researchers who use animals, as our results indicate that neither the ARRIVE nor the GSPC guidelines were used by such researchers from 2010 July to 2012. The ARRIVE and GSPC guidelines outline the minimum information that should be provided when reporting an animal experiment(s). These guidelines should be used when designing, performing, reporting, reviewing, and publishing an animal study. They will also be of use to funding organizations, especially large funding bodies, which have a responsibility to provide financial assistance and to strictly evaluate the quality of the research that is funded[18]. Good reporting allows readers to fully understand the methods and processes involved and to assess the reliability and validity of the findings.

There are several limitations to our study. First, our analyses were limited to studies on rodents, and studies on other animals were excluded. Second, we only included animal experiments involving neoplasms that were published in Chinese journals between 2010 and 2012. Third, our scoring criteria (yes or no) did not allow for partial information. Fourth, we used unweighted scores for the ARRIVE checklist items, although this may not be a valid approach. Finally, in general, the reporting quality of experiments involving neoplasms in animals in Chinese journals may be worse than what we found because all the studies included in our analyses were supported by the NSFC.

Supporting Information

Acknowledgments

We thank Xiaohuan Feng, Lili Zhang, Xiaokang Zhang, Xin Tian, Yimin Lu, and Qingshan Guo (Lanzhou University) for previous contributions to this study. We thank BiomEditor and American Journal Experts (AJE) for providing assistance with the final revisions of the manuscript.