Figures
Abstract
Introduction
Although selective reporting of clinical trial results introduces bias into evidence based clinical decision making, publication bias in pediatric epilepsy is unknown today. Since there is a considerable ambiguity in the treatment of an important and common clinical problem, pediatric seizures, we assessed the public availability of results of phase 3 clinical trials that evaluated treatments of seizures in children and adolescents as a surrogate for the extent of publication bias in pediatric epilepsy.
Methods
We determined the proportion of published and unpublished study results of phase 3 clinical trials that were registered as completed on ClinicalTrials.gov. We searched ClinicalTrials.gov, PubMed, and Google Scholar for publications and contacted principal investigators or sponsors. The analysis was performed according to STROBE criteria.
Results
Considering studies that were completed before 2014 (N = 99), 75 (76%) pediatric phase 3 clinical trials were published but 24 (24%) remained unpublished. The unpublished studies concealed evidence from 4,437 patients. Mean time-to-publication was 25 SD ± 15.6 months, more than twice as long as mandated.
Citation: Lampert A, Hoffmann GF, Ries M (2016) Ten Years after the International Committee of Medical Journal Editors’ Clinical Trial Registration Initiative, One Quarter of Phase 3 Pediatric Epilepsy Clinical Trials Still Remain Unpublished: A Cross Sectional Analysis. PLoS ONE 11(1): e0144973. https://doi.org/10.1371/journal.pone.0144973
Editor: Silvio Garattini, Mario Negri Institute for Pharmacology Research, ITALY
Received: August 31, 2015; Accepted: October 26, 2015; Published: January 6, 2016
Copyright: © 2016 Lampert et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper.
Funding: AL received a personal scholarship from "Dr. August and Dr. Anni Lesmüller foundation." The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: A. Lampert reports grants from "Dr. August and Dr. Anni Lesmüller foundation," during the conduct of the study; personal fees and non-financial support from Chamber of Pharmacists, Baden-Württemberg, personal fees from European Society of Clinical Pharmacy, outside the submitted work; Dr. Hoffmann has nothing to disclose. Dr. Ries reports grants and personal fees from Shire, personal fees from GSK, personal fees from Oxyrane, personal fees from Alexion, outside the submitted work. This does not alter the authors' adherence to PLOS ONE policies on sharing data and materials.
Introduction
“In the nineteenth century health was transformed by clear, clean water. In the twenty-first century, health will be transformed by clean clear knowledge.”
Sir Muir Gray
It is estimated that 30–50% of all the clinical trials that have been conducted and completed are still not published in academic journals [1–5]. However, publication bias in pediatric epilepsy has not been investigated. Epilepsy is a relatively frequent, serious condition associated with high morbidity for patients [6]. Pediatric seizures can have a deleterious impact on a child’s development causing disability and lifelong dependency [7]. In addition, seizures, hospitalizations, emergency department visits, or medication burden disrupt lives of patients and afflicted families [7]. Clinically available anti-epileptic drugs fail to control seizures in approximately 30% of epileptic patients [8, 9]. Beyond pharmacoresistance the long-term use of anti-epileptic drugs is limited by adverse events, drug-drug-interactions, and non-compliance due to inconvenient regimens [10–12].
Pediatric treatment decisions are often based on incomplete clinical data and are characterized by off-label use due to lack of clinical trials in children [13]. Waste of knowledge due to incomplete publication of trial results impedes complete assessment of the effect of an intervention [14]. Indeed, outcome data that favor the efficacy of the investigated drug are twice as likely to be published [1, 15, 16]. Consequently, when unfavorable results of drug trials are not published, the efficacy of a drug may be overestimated and trials may be unnecessarily repeated which consequently wastes resources. Considering insufficient clinical data in pediatrics, publication bias can particularly distort the apparent efficacy of a drug which complicates the interpretation of medical literature and decision making about an individual treatment [15, 17, 18]. Of note, several historical examples demonstrate that retention of findings especially concerning adverse events seriously impairs treatment decisions [19–21]. For example, the retention of reporting increased mortality rates during clinical trials with the antiarrhythmic drug lorcainide in 1980, concealed early warnings regarding the risk for cardiac death [21,22]. Beyond the impact on treatment decisions which affects all patients, there is an explicit ethical obligation to publish towards study participants mandated by the Declaration of Helsinki. Patients participate in clinical research on the understanding that findings will be of public interest. Therefore, non-publication of trial outcome data violates an ethical obligation that investigators have towards study participants [23,24].
Phase 3 clinical trials are usually rigorously designed, comparing the investigational compound against placebo or an active control in the population of interest and thus, provide a high level of evidence [25]. In addition, systematic reviews and meta-analysis investigating treatment options and treatment guidelines are largely based on such trials [17]. Therefore, publication bias of phase 3 clinical trials can have a pronounced impact on therapeutic decisions. Since the current publication bias in phase 3 clinical trials in children and adolescents with epilepsy is unknown, we assessed the public availability of phase 3 clinical trials addressing treatments of seizures in children and adolescents as a surrogate for the extent of publication bias in pediatric epilepsy. This analysis aimed at highlighting the current publication bias facilitating more cautious interpretation of current knowledge in pediatric epilepsy treatment decisions.
Methods
The analysis was performed according to STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) criteria.
ClinicalTrial.gov query
For cross sectional analysis, we searched ClinicalTrials.gov on February 3, 2015 with the key word ‘epilepsy’ and restricted search to completed phase 3 clinical trials with children. ClinicalTrials.gov defines the recruiting status as completed when “the clinical study has ended normally, and participants are no longer being examined or treated (i.e, the "last subject, last visit" has occurred)”. The search output was downloaded in a spreadsheet format representing the basis for analysis. For the analysis all completed phase 3 clinical trials were included that were registered on ClinicalTrials.gov investigating the treatment of pediatric seizures, irrespective of the interventions studied (e.g., surgical procedures or drugs). In addition, terminated phase 3 clinical trials with children investigating treatments for epilepsy were analyzed for publication status and reasons for termination provided on ClinicalTrials.gov. ClinicalTrials.gov defines the recruiting status as terminated when “the clinical study has stopped recruiting or enrolling participants early and will not start again. Participants are no longer being examined or treated. As such, a terminated trial does not represent a fully completed data collection available for publication and thus, terminated trials were analyzed separately. A trial was considered as published when results were presented in the ClinicalTrials.gov record or when a journal had published a peer-reviewed manuscript online or in print, which included any outcome data from the trial in question. In a sensitivity analysis, we took an additional conservative approach to the publication expectation and assessed trials completed before 2014 separately in accordance with the Food and Drug Administration Amendments Act (FDAAA) of 2007 that requires publication within one year after completion of the trial [26].
Publication search
When the ClinicalTrial.gov record did not provide results we searched for manuscripts by reviewing the ClinicalTrials.gov record for references to published outcome data. ClinicalTrial.gov encourages investigators to provide a link to PubMed indexed manuscripts that contain results from the registered trial. In addition, ClinicalTrial.gov uses the unique trial identification number (NCT number) to automatically identify and link corresponding PubMed entries. For trials not providing a corresponding PubMed link we searched PubMed and Google Scholar for the trials in question. Search terms for PubMed and Google Scholar included the NCT number, other study ID numbers listed in the ClinicalTrials.gov record, the investigated drug, ‘epilepsy’ or the specified condition, and details of the study design (e.g., ‘double blind’, ‘randomized’, or inclusion and exclusion criteria). Registry entries and publications identified by the PubMed and Google Scholar search were matched by consensus based on the following study characteristics: study title, investigated drug, study design, number of participants, and inclusion and exclusion criteria. If no published results of registered trials could be identified, the principal investigator or sponsor listed in the ClinicalTrials.gov record was contacted.
Time-to-publication.
We calculated the time-to-publication as the number of months from the primary completion date of the trial and the publication of results either on ClinicalTrials.gov or in a peer-reviewed Journal. ClinicalTrials.gov defines the primary completion date as “the date that the last participant in a clinical study was examined or received an intervention and that data for the primary outcome measure were collected.[…] The primary completion date is the term used on ClinicalTrials.gov for "completion date" defined in Section 801 of the Food and Drug Administration Amendments Act of 2007”. If the primary completion date was not available the completion date of the trial was used. The calculation of time-to-publication was limited to studies completed before 2014 in accordance with the FDAAA that requires publication within one year after completion of the trial [26].
Statistical analysis
The following continuous or categorical variables were analyzed: NCT number, study title, gender, age, study phase, study type, study design, condition, intervention, recruitment status, primary completion date and completion date, availability of study results, publication date, time-to-publication, sponsor, and funding source. Standard methods of descriptive statistics were applied. All calculations were performed with SAS Enterprise Guide version 5.1 (SAS, Cary, NC, USA).Two-sided p values ≤0.05 were considered statistically significant.
Results
Pediatric phase 3 clinical trials investigating antiepileptic treatment
The ClincalTrials.gov search initially identified 117 completed trials (Fig 1). We subsequently excluded eight trials from the analysis because they did not investigate epilepsy and one trial because no children or adolescents were enrolled. Therefore, a total of 108 completed phase 3 clinical trials investigating anti-epileptic treatment in children and adolescents registered on ClinicalTrials.gov were searched for publication of results. The year of completion ranged from 1996 to 2014 (Fig 2). The completion date of six studies remained unknown whereof three studies were published in 2007 and 2009 and thus, considered for analysis as completed before 2013. A total of 15 terminated phase 3 pediatric epilepsy clinical trials were identified (Table 1). Reasons for termination were multifaceted (e.g., recruitment challenges, unlikely attaining a positive outcome, or Sponsor’s decision).
A) Number of trials by year of completion. B) Number of enrolled patients by year of completion.
Publication status—trials, participants, and time-to-publication
Overall, 76 (70%) completed studies were published and 32 (30%) completed studies have remained unpublished. Considering only studies that were completed before 2014 (N = 99), which was the current deadline mandated by the FDAAA, that requires publication of results within one year after completion of the trial, 75 (76%) completed studies were published and 24 (24%) remained unpublished. A total of 27,587 patients were enrolled in the identified completed studies (N = 107 studies with available data) (Table 2). Up to date, data involving 6,464 participants are not yet publically available and 4,437 participants when considering studies completed before 2014. The mean time-to-publication, i.e. the delay from completion of the trial until public availability of the outcome data, was 25 SD ± 15.6 months (median 22, IQR 11 to 34, N = 70 trials with available data) for studies that were completed before 2014 (Fig 3). Overall 33% (23/70) studies were published within a year as mandated by FDAAA. Considering published studies, time-to-publication showed a decreasing trend over the years. When considering the mean publication time of 25 months and thus analyzing studies completed before 2013 (N = 92), 71 (77%) studies were published and 21 (23%) remained unpublished. Terminated trials provided results for four studies on clinicaltrials.gov.
“FDAAA” indicates the timeline mandated by the Food and Drug Administration Amendments Act of 2007.
Compounds investigated, demography, and funding
The identified completed pediatric phase 3 clinical trials investigated various anti-epileptic interventions (Table 3). Thereby, the most investigated anti-epileptic treatment was levetiracetam with 17 published trials and six unpublished trials. Studies investigating brivaracetam remained more often unpublished than published. Brivaracetam, lecetiracetam, and topiramate had more than three unpublished studies. The majority of the identified pediatric trials included both genders, and had enrolled also adults and elderly patients in addition to children (Table 4). Twenty-one percent (5/23) trials including solely children remained unpublished. The majority of identified trials were funded by pharmaceutical industry.
Discussion
Almost every fourth completed phase 3 clinical trial that investigates anti-epileptic interventions in children and adolescents remains unpublished. Consequently, results from 4,437 study participants are not available for clinical decision-making when considering unpublished trials completed before 2014. This induces a considerable publication bias into the assessment of efficacy and safety of interventions intended to treat pediatric seizures. As a first measure to address publication bias, ten years ago the International Committee of Medical Journal Editors (ICMJE) has required that prospective trials involving human participants must be registered prior to the beginning of study enrollment in order to be considered for publication in member journals [27]. This requirement was later incorporated into the CONSORT statement for reporting clinical trials [28]. However, although the ICMJE requires trial registration as a precondition for publication, every fourth published trial is still not registered [29]. Therefore, there might be an unknown number of clinical trials that have been conducted without registration and consequently without publication of the results. The mean time-to-publication of the anti-epileptic pediatric phase 3 trials in the present study was 25 months which is more than twice as long as the time mandated by the FDAAA. In 2007 the prospective registration and mandatory publication of applicable trials (i.e., other than phase 1 clinical investigations, drugs with FDA approval, and at least one study site in the United States) within one year of completion became federal law in the United States with the FDAAA [26]. However, most trials do not comply with mandatory timely reporting [30]. In addition, this law is restricted to trials conducted in the US and requires only publication of trials that are completed after 2008. However, health care providers rely most of the time on evidence from trials done earlier to make treatment decisions. In particular, most of the anti-epileptic drugs used in pediatrics were approved several years ago, and thus, the rates of missing data may have a detrimental impact on current clinical practice when treating pediatric seizures. Therefore, the AllTrials initiative (www.alltrials.net) again calls for registration and publication of all results of all clinical trials—past and future—on all treatments in current use. Of interest, the publication pattern between published and unpublished reports was similar for all completed studies at any year and studies completed at certain cut-off years in a sensitivity analysis considering a) studies completed before 2014 (taking into account the FDAAA timeline of one year), and b) studies completed before 2013 (taking into account the mean time to publication of 25 months). Studies investigating treatments for epilepsy in children are particularly confronted with recruitment challenges leading to premature termination of studies. Recruitment challenges are a well-known barrier to pediatric clinical trials in general and thus, improving research infrastructure and consequent registration of studies to inform practitioners are needed to compound the problem [31]. Particularly pediatric societies should encourage members to improve the rate of independent studies and transparency of results and also to follow-up the benefit of approved drugs in pediatrics. There are no excuses for not publishing trial results because a low-threshold way to get any results published whether they are positive or negative is posting on a clinical trial registry, such as ClinicalTrials.gov. Researchers might fear to not getting the data published afterwards in a peer-reviewed journal. However, the ICMJE even encourages publication of clinical trial results in clinical trial registries and does not consider the posting of results as a barrier to publication in a member journal [32]. Terminated trials remained most likely unpublished due to incomplete data collection. Of interest, most phase 3 clinical trials involving children and adolescents with epilepsy are sponsored by the industry, therefore it is not surprising that most unpublished studies are industry-sponsored, too. This is nevertheless important taking into account the recent findings revealing that industry complies better with results reporting than NIH or other government or academic institution [33].
This study has several limitations. Since ClinicalTrials.gov is considered the most relevant clinical trial registry we did not investigate other databases (e.g., the German Clinical Trials Register or the Australian New Zealand Clinical Trials Registry). In addition, the investigation of a clinical trial registry implies that only registered trials were included in our analysis. In order to prevent misclassifying a trial as unpublished, we conducted an exhaustive literature search in two major databases (i.e., PubMed and GoogleScholar) with multiple search terms and contacted investigators or sponsors. Our study did not formally assess whether the content of the publications was consistent with the original research questions and the pre-specified statistical analysis plan since this information was not completely publically available for all trials. Furthermore, we did not formally monitor study sites or study sponsors whether the reported information in the public domain was correct. This analysis assumes that the entries provided on ClinicalTrials.gov are accurate and complete as mandated by the FDAAA [26]. Although the ClinicalTrials.gov allows examination of various aspects of ongoing and completed clinical trials, its ultimate usefulness depends on the research community to submit accurate, informative data.
Our data define the current publication bias in phase 3 clinical trials investigating anti-epileptic interventions ten years after the ICMJEs’ clinical trial registration initiative and represent today’s baseline for the future. Regular follow-ups will be of interest in order to document the impact of legal requirements and initiatives such as the AllTrials initiative on investigator’s compliance with publication of clinical trial outcome data. We hope that the publication efforts will increase over time in order to allow key-stakeholders more informed decisions based on true evidence for the benefit of children and adolescents with epilepsy.
Author Contributions
Conceived and designed the experiments: MR AL. Performed the experiments: MR AL. Analyzed the data: MR AL GH. Contributed reagents/materials/analysis tools: MR AL GH. Wrote the paper: MR AL GH.
References
- 1. Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health technology assessment. 2010;14(8):iii, ix–xi, 1–193. pmid:20181324.
- 2. Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS medicine. 2009;6(9):e1000144. pmid:19901971; PubMed Central PMCID: PMC2728480.
- 3. Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of large randomized clinical trials: cross sectional analysis. Bmj. 2013;347:f6104. pmid:24169943; PubMed Central PMCID: PMC3812466.
- 4. Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis. Bmj. 2012;344:d7292. pmid:22214755; PubMed Central PMCID: PMC3623605.
- 5. Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Annals of internal medicine. 2010;153(3):158–66. pmid:20679560; PubMed Central PMCID: PMC3374868.
- 6. Nevalainen O, Ansakorpi H, Simola M, Raitanen J, Isojarvi J, Artama M, et al. Epilepsy-related clinical characteristics and mortality: a systematic review and meta-analysis. Neurology. 2014;83(21):1968–77. Epub 2014/10/24. pmid:25339211.
- 7. Berg AT, Baca CB, Loddenkemper T, Vickrey BG, Dlugos D. Priorities in pediatric epilepsy research: improving children's futures today. Neurology 2013;81:1166–75. pmid:23966254
- 8. French JA. Refractory epilepsy: clinical overview. Epilepsia. 2007;48 Suppl 1:3–7. pmid:17316406.
- 9. Potschka H, Brodie MJ. Pharmacoresistance. Handbook of clinical neurology. 2012;108:741–57. Epub 2012/09/04. pmid:22939063.
- 10. Marson AG, Kadir ZA, Hutton JL, Chadwick DW. The new antiepileptic drugs: a systematic review of their efficacy and tolerability. Epilepsia. 1997;38(8):859–80. pmid:9579887.
- 11. Greenwood RS. Adverse effects of antiepileptic drugs. Epilepsia. 2000;41 Suppl 2:S42–52. pmid:10885739.
- 12. Cramer J, Vachon L, Desforges C, Sussman NM. Dose frequency and dose interval compliance with multiple antiepileptic medications during a controlled clinical trial. Epilepsia. 1995;36(11):1111–7. pmid:7588455.
- 13.
EMA. Assessment of the paediatric needs epilepsy 2006 [cited 2015 15 Apr]. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Other/2009/10/WC500004047.pdf
- 14. Glauser T, Ben-Menachem E, Bourgeois B, Cnaan A, Chadwick D, Guerreiro C, Kalviainen R, Mattson R, Perucca E, Tomson T. ILAE treatment guidelines: evidence-based analysis of antiepileptic drug efficacy and effectiveness as initial monotherapy for epileptic seizures and syndromes. Epilepsia 2006;47:1094–120. pmid:16886973
- 15. Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS medicine. 2008;5(11):e217; discussion e. pmid:19067477; PubMed Central PMCID: PMC2586350.
- 16. Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. Bmj. 2003;326(7400):1171–3. pmid:12775615; PubMed Central PMCID: PMC156459.
- 17. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. Bmj. 2012;344:d7202. pmid:22214754.
- 18. Whittington CJ, Kendall T, Fonagy P, Cottrell D, Cotgrove A, Boddington E. Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet. 2004;363(9418):1341–5. pmid:15110490.
- 19. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. The New England journal of medicine. 2008;358(3):252–60. pmid:18199864.
- 20. Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. The New England journal of medicine. 2007;356(24):2457–71. pmid:17517853.
- 21. Cowley AJ, Skene A, Stainer K, Hampton JR. The effect of lorcainide on arrhythmias and survival in patients with acute myocardial infarction: an example of publication bias. International journal of cardiology. 1993;40(2):161–6. pmid:8349379.
- 22. Echt DS, Liebson PR, Mitchell LB, Peters RW, Obias-Manno D, Barker AH, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The Cardiac Arrhythmia Suppression Trial. The New England journal of medicine. 1991;324(12):781–8. pmid:1900101.
- 23. Pearn J. Publication: an ethical imperative. Bmj. 1995;310(6990):1313–5. pmid:7773049; PubMed Central PMCID: PMC2549684.
- 24. Munro AJ. Publishing the findings of clinical research. Bmj. 1993;307(6915):1340–1. pmid:8257891; PubMed Central PMCID: PMC1679429.
- 25. Guyatt GH, Haynes RB, Jaeschke RZ, Cook DJ, Green L, Naylor CD, et al. Users' Guides to the Medical Literature: XXV. Evidence-based medicine: principles for applying the Users' Guides to patient care. Evidence-Based Medicine Working Group. Jama. 2000;284(10):1290–6. pmid:10979117.
- 26.
FDAAA. Sec. 801. Expanded clinical trial registry data bank 2007 [cited 2015 21 Apr]. Available from: http://www.gpo.gov/fdsys/pkg/PLAW-110publ85/html/PLAW-110publ85.htm.
- 27. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. The New England journal of medicine. 2004;351(12):1250–1. pmid:15356289.
- 28. Schulz KF, Altman DG, Moher D, Group C. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. PLoS medicine. 2010;7(3):e1000251. pmid:20352064; PubMed Central PMCID: PMC2844794.
- 29. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. Jama. 2009;302(9):977–84. pmid:19724045.
- 30. Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. Bmj. 2012;344:d7373. pmid:22214756.
- 31. Tishler CL, Reiss NS. Pediatric drug-trial recruitment: enticement without coercion. Pediatrics 2011;127:949–54. pmid:21464193
- 32.
ICMJE. Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals 2014 [cited 2015 21 Apr]. Available from: http://icmje.org/icmje-recommendations.pdf
- 33. Anderson ML, Chiswell K, Peterson ED, Tasneem A, Topping J, Califf RM. Compliance with results reporting at ClinicalTrials.gov. The New England journal of medicine. 2015;372(11):1031–9. pmid:25760355.