Figures
Abstract
Background
Recent systematic reviews of randomised controlled trials (RCTs) in relapsing multiple sclerosis (RMS) revealed a decrease in placebo annualized relapse rates (ARR) over the past two decades. Furthermore, regression to the mean effects were observed in ARR and MRI lesion counts. It is unclear whether disease progression measured by the expanded disability status scale (EDSS) exhibits similar features.
Methods
A systematic review of RCTs in RMS was conducted extracting data on EDSS and baseline characteristics. The logarithmic odds of disease progression were modelled to investigate time trends. Random-effects models were used to account for between-study variability; all investigated models included trial duration as a predictor to correct for unequal study durations. Meta-regressions were conducted to assess the prognostic value of a number of study-level baseline variables.
Results
The systematic literature search identified 39 studies, including a total of 19,714 patients. The proportion of patients in placebo controls experiencing a disease progression decreased over the years (p<0.001). Meta-regression identified associated covariates including the size of the study and its duration that in part explained the time trend. Progression probabilities tended to be lower in the second year of a study compared to the first year with a reduction of 28% in progression odds from year 1 to year 2 (p = 0.017).
Citation: Röver C, Nicholas R, Straube S, Friede T (2015) Changing EDSS Progression in Placebo Cohorts in Relapsing MS: A Systematic Review and Meta-Regression. PLoS ONE 10(9): e0137052. https://doi.org/10.1371/journal.pone.0137052
Editor: Orhan Aktas, University of Düsseldorf, GERMANY
Received: May 13, 2015; Accepted: August 12, 2015; Published: September 1, 2015
Copyright: © 2015 Röver et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: Data were extracted from publically available papers.
Funding: RN is grateful for support from the NIHR Biomedical Research Centre. The authors received no specific funding for this work.
Competing interests: CR has declared that no competing interests exist. RN has received honoraria for speaking fees for advisory boards or search funds from Bayer, Biogen, Genzyme, Merck Serono, Novartis, Roche, TEVA and NICE. SS declares honoraria from Oxford Medical Knowledge and advisory board fees from Daiichi Sankyo, Inc. TF is a consultant to Novartis, Biogen, Grünenthal, Bayer, Vertex, Janssen, and AstraZeneca. This does not alter the authors’ adherence to PLOS ONE policies on sharing data and materials.
Introduction
Recent systematic reviews of placebo groups in randomised controlled trials (RCTs) in relapsing multiple sclerosis (RMS), i.e. relapsing remitting and relapsing secondary progressive multiple sclerosis, suggest a decrease in annualized relapse rates (ARRs) over calendar time [1–3] as well as a decrease in relapse rates over the course of the study [4]. These changes appear to relate to the changing eligibility criteria and populations entering trials and regression to the mean effects [3]. Pre-trial ARR and mean baseline age were independently identified as predictors for on-trial ARR in a smaller number of phase III trials with at least 18 months follow-up by Stellmann et al. [5]. Recently regression to the mean in MRI lesion counts was identified and quantified in a systematic review and meta-analysis [6].
Disability outcomes in multiple sclerosis (MS) are a key component that regulators have identified as the principal target for an increasing range of therapies targeting the underlying disease process [7]. The commonly used method of disability measurement in trials is the extended disability status scale (EDSS) [8].
In this paper we aim to investigate whether a decrease in placebo ARRs observed in randomized controlled trials in RMS is also present in EDSS progression and, if so, whether it can also be explained by changes in patient populations and study design characteristics. Furthermore, we will assess placebo controls of RCTs for regression to the mean effects in EDSS progression.
Methods
Systematic literature search
A recently conducted systematic literature search for placebo controlled randomised trials in RMS [3] was updated by searching PubMed with the aim of identifying placebo-controlled, double-blind RCTs in MS where all or some of the patients had a relapsing form of the disease and that reported data on pre-trial and on-trial ARR as well as on pre-trial and on-trial EDSS. To update the previous systematic review we searched for articles published from 2011 onwards with the search terms “multiple sclerosis”, “relapse rate” and “placebo”. All abstracts were independently screened by two reviewers. The search was performed on February 20th, 2015. If one reviewer suggested that the full paper should be examined after reading the abstract, the full paper was considered. For a trial to be included in this systematic review it had to be randomised, double blind, and placebo-controlled, with at least some of the trial participants having RMS and results for participants with RMS having been reported seperately. Trials had to assess the efficacy of disease modifying drugs (i.e. not assessing symptomatic therapies), and report data on both clinical relapses and EDSS. We excluded cross-over trials and studies where patients in the control group received a form of active treatment (add-on therapy).
Data extraction
The following data were extracted by one reviewer and verified by another:
- publication date, treatment groups and corresponding numbers of patients, and duration of follow-up,
- the proportion of patients experiencing a worsening in EDSS,
- the ordinates at years 1 and 2 of Kaplan-Meier curves of confirmed EDSS progression, or equivalent tabulated data.
Data analysis
For the purpose of all analyses of temporal trends, the year and month of publication and the study durations were used. EDSS progression was analyzed based on the logarithmic odds (log-odds) of disease progression; these result as , where
is the fraction of progressing patients. Corresponding standard errors are calculated based on a binomial model as
, where N is the total number of patients, out of which k have progressed. We assessed publication bias using funnel plots and Egger’s tests. In the meta-regression analyses we utilized linear regression methods, accounting for the standard errors of the individual studies. Random-effects models were used in order to account for potential between-study variability, and all investigated models included trial duration as a predictor in order to correct for unequal study durations [9]. Between-study heterogeneity τ2 was estimated using the Mandel-Paule method [10] which is reported with the p-values of the chi-square test of heterogeneity. Furthermore, the I2 measure for between-trial heterogeneity is reported with a 95% confidence interval. For the multiple regression, we used the Bayesian information criterion (BIC) to determine the best-fitting model among all possible subsets of predictors [11]. For the predicted means 95% confidence intervals were calculated. We used random-effects meta-analysis on the log odds scale to investigate the probability of EDSS progression during first and second year of a study. EDSS progression probabilities for years 1 and 2 were calculated from the cumulative probabilities reported in the studies. In estimating the standard errors of the log risk ratios we neglected correlations between the risk estimates (the empirical fractions of patients are positively correlated, and so we err on the conservative side here, overestimating the uncertainty in differences or ratios leading to wider confidence intervals). We used the R software (www.r-project.org) in conjunction with the metafor package (http://cran.r-project.org/package=metafor) to perform meta-analyses and meta-regressions.
Results
Studies identified and clinical features of the placebo groups
The systematic literature search identified 39 studies, including a total of 19,714 patients, of which 6,947 received placebo. The cumulative observation time amounts to 31,368 patient-years, with a contribution of 11,163 patient-years from placebo treated patients. The study selection process is illustrated in the flow chart in S1 Fig. In the placebo controls of the identified trials the medians of the mean baseline EDSS and pre-trial ARR were 2.7 (range 1.9 to 5.1) and 1.4 (range 0.9 to 2.1), respectively. During the trials the placebo ARR were somewhat lower with a median of 0.8 (range 0.22 to 1.8). The study characteristics are summarized in Table 1. Disease progression can be defined in a number of ways, of the 39 included studies, disability worsening of one point or more on the EDSS scale was used in 34 (87%) and progression confirmation of 3 months or more was used in 23 (59%).
N denotes the number of treatment arms for which the corresponding figures could be extracted.
Decreasing placebo EDSS progression rates over the past two decades
Fig 1 shows the fractions of progressing patients as reported in different studies’ placebo groups over the years. The chances of progression of course also depend on the trial duration (“shorter” and “longer” studies are also indicated by different symbols), but even after accounting for the individual follow-up times, the regression analysis indicates a statistically significant effect of the publication year with the odds for disease progression decreasing by 31% (95% CI [17%, 42%]) within a decade (p<0.001; between-trial heterogeneity τ2 = 0.15 (95% CI [0.07, 0.31]), I2 = 79.8% (95% CI [66.3, 89.2]), p<0.001). The funnel plots (not shown) and Egger’s tests (p = 0.22 for short studies and p = 0.45 for long studies) did not suggest any publication bias.
The chances of progression also depend on the study duration (you can see that shorter studies have smaller fractions) but even after accounting for the duration, the decreasing trend remains statistically significant (p<0.0001). The red line shows the estimated regression line for a trial duration of 1 year.
Predictors of EDSS disease progression
Table 2 shows the results from univariate regression analyses using the remaining covariates (excluding the publication year). These are the number of placebo patients, study length, Oxford Quality Scale score, use of confirmed progression, the number eligibility criteria as well as the number of words used to describe them, mean baseline age in years, mean baseline EDSS, and baseline ARR, mean baseline MS duration in years, follow-up duration in years, and number of treatment arms. For each of the predictors investigated, the odds ratio, which is the multiplicative change in odds for EDSS progression for every unit increase in the predictor, is shown along with its 95% confidence interval and p-value. Note that all investigated models included follow-up time as a covariate. For each analysis, the value of τ2, the random effect accounting for between-study heterogeneity is shown as well, along with the relative reduction in τ2 compared to the model that only uses follow-up time as a predictor. With larger numbers of placebo patients the progression probability decreases (OR 0.998 per patient, 95% CI [0.998, 0.999], p = 0.0004). As would have been expected, the progression probability increases with the length of the study (OR 1.852 for studies longer than 1 year vs. studies up to 1 year, 95% CI [1.09, 3.16], p = 0.023 for < = 1 vs. >1 year; OR 1.61 per year, 95% CI [1.31, 1.98], p<0.0001 for linear trend).
Regression coefficients relate to the logarithmic odds of progression. τ2 denotes the unexplained between-study heterogeneity, and the reduction percentages relate to the model including only follow-up duration as a predictor (which is also included in all univariable models). The multivariable model was selected based on the Bayesian information criterion (BIC).
The model selection result is also shown in Table 2. The variables included in the final model relate to study size (number of placebo patients: OR 0.997, 95% CI [0.996; 0.999], p<0.001), study duration (follow-up duration in years: OR 1.17 per year, 95% CI [0.889, 1.54], p = 0.26; study longer 1 year: OR 1.88, 95% CI (1.16, 3.16), p = 0.018) and baseline disease status (baseline ARR: OR 0.883, 95% CI [0.477, 1.63], p = 0.69; mean baseline EDSS: OR 0.822 per point on EDSS scale, 95% CI [0.619, 1.09], p = 0.18) and confirmation of disease progression (OR 1.37 for confirmation vs. without confirmation, 95% CI [0.799, 2.35], p = 0.25).
Decreasing EDSS disease progression over follow-up time
Fig 2 illustrates the probabilities of progression during the first and second years of follow-up for the 13 studies where relevant data could be extracted. Overall progression probabilities tend to be lower in the second year, with the exception of two small studies from the 1990’s. The combined odds ratio comparing the progression probabilities from year 2 to year 1 is 0.72 (95% CI [0.56; 0.93], p = 0.017; between-trial heterogeneity τ2 = 0.096 (95% CI [0.017; 0.40], I2 = 57.7% (95% CI [19.2; 85.1]), p = 0.0028) which translates to a reduction of 28% in progression odds from year 1 to year 2. In the S2 Fig, a forest plot of the progression odds ratios is given providing a more conventional summary of the effects. Looking at combined progression probabilities from random-effects meta-analyses, chances are 16.9% during the first year, and 13.1% during the second year. These studies used confirmed disease progression as the endpoint. When considering only the eight most recent studies published during the last decade, the numbers change slightly to 16.0% and 11.2% during first and second year, respectively.
Connecting lines indicate the rates for the two subsequent years, line widths are proportional to study sizes (numbers of patients N). The weighted average (weighted by the inverse variances on the log odds scale) decreases from 16.9% to 13.1% from first to second year. [16.0% to 11.2% for the 8 most recent post-2000 studies].
Discussion
For the key clinical outcome measure of disease progression in MS [7], this study has confirmed that in relapsing MS trial placebo groups the rates of disease progression reduce with the more recent publication year of the study and between the first and second year of the study. These findings further question the appropriateness of comparing relative risk reductions of drug to placebo for newer to older drugs in an attempt to argue higher potency of the newer agents. Recently a review including more than 1,000 meta-analyses from the Cochrane Library revealed a decrease in publication bias over time [12], but we feel this is unlikely to be the explanation of our findings as publication bias is usually associated with the treatment effect and we only considered the outcome of the control group.
Despite meta-regressions with this number of studies having a limited power, we found that increasing study size and decreasing length of the study were significant in reducing rates of disease progression seen in studies with later publication year. Study size has already been shown to be highly correlated with publication year for ARR [3]. However, a reduced ARR with later publication date was associated with a shorter study length whereas an increased rate of EDSS progression rate was associated with increased study length. For ARR this is because the earlier studies were longer and the study populations of the earlier trials had higher disease activity whereas for disease progression the increase with length of study is due to the way in which cumulative progression is calculated, i.e. those who progress in the first year are added to those who progress subsequently. Thus both these features may well be simply confounding factors and, unlike in the case of ARR, we did not find any other subject or study design features explaining this trend [3]. With the advance in diagnostics (e.g. MRI) and therapies in RMS over the past two decades, the eligibility criteria for RMS trials changed considerably over the years. In our analyses we captured the number of eligibility criteria and the number of words used to describe these. We did not carry out any detailed analyses of the eligibility criteria, which could be considered a limitation of our analysis and could be the subject of future research.
We have also found that the rates of disease progression reduce from the first to the second year implying as with ARR rates there is a regression to the mean effect. This is seen within the context of RMS where subjects are chosen to enter studies after a series of relapses which then can settle down spontaneously [4]. However, as with the time-to-first relapse endpoint an alternative explanation for the apparent time-dependence in the time-to-disease progression data could be between-patient heterogeneity [13]. The observed trend warrants further investigation in the form of analyses of individual patient data (IPD). IPD meta-analyses would allow the investigation of both study-level and patient-level characteristics.
Although correlations on a patient level between MRI outcomes, relapse and EDSS progression are small [14,15], known as the clinico-radiological paradox [16], the trends over time on a population level are quite similar in these three measures. Given the similar relationship what we may be seeing is the impact of the early phase of MS relapses and MRI activity on the disability score. The EDSS has acknowledged limitations [17–19] and levels of disability especially in early MS are very variable [20] as relapses are known to bias the assessment of disability. Here disability progression was defined predominantly using a one point increase in the EDSS combined with 3 months confirmation. Time to 3 months confirmed disability progression is known to be more susceptible to the impact of relapses than measures that use a longer period to confirm a change in disability [7,19]. In summary, EDSS disease progression as used here exhibits similar behaviour over time as the ARR and point to changes in trial characteristics over the years, questioning comparisons between historical and recent trials.
In our investigations we have paid careful attention to modelling potential between-study heterogeneity by using the Mandel-Paule estimator [10]. In the light of recent findings suggesting that between-study heterogeneity is often underestimated, in particular in meta-analyses with small numbers of trials, [21] this appears to be appropriate. At least in the main analyses presented here, however, the number of trials was sufficiently large to estimate the between-study variation reliably.
Investigations into the prediction of treatment response are an ongoing effort with some promising approaches including the modified Rio score [22,23]. These might allow a stratification of trial populations in the future resulting ultimately in a better understanding of which patients benefit most from certain therapies.
Supporting Information
S1 PRISMA Checklist. PRISMA checklist of items to include when reporting a systematic review or meta-analysis.
https://doi.org/10.1371/journal.pone.0137052.s001
(DOCX)
S1 Data File. Data file containing the data extracted from the publications included in the meta-regression.
https://doi.org/10.1371/journal.pone.0137052.s002
(CSV)
S2 Data File. Data file containing the data extracted from the publications included in the meta-analysis comparing progression probabilities in year 1 and year 2.
https://doi.org/10.1371/journal.pone.0137052.s003
(CSV)
S1 Fig. The PRISMA flow chart illustrating the systematic literature review.
https://doi.org/10.1371/journal.pone.0137052.s004
(EPS)
S2 Fig. Forest plot of the progression probability ratio with random effects (RE) meta-analysis.
The numbers in brackets are 95% confidence intervals.
https://doi.org/10.1371/journal.pone.0137052.s005
(EPS)
Author Contributions
Conceived and designed the experiments: TF. Performed the experiments: CR RN SS. Analyzed the data: CR TF. Wrote the paper: CR RN SS TF.
References
- 1. Inusah S, Sormani MP, Cofield SS, Aban IB, Musani SK, Srinivasasainagendra V, et al. Assessing changes in relapse rates in multiple sclerosis. Multiple Sclerosis. 2010;16(12):1414–1421. pmid:20810517
- 2. Nicholas R, Straube S, Schmidli H, Schneider S, Friede T. Trends in annualized relapse rates in relapsing-remitting multiple sclerosis and consequences for clinical trial design. Multiple Sclerosis Journal. 2011;17(10):1211–1217. pmid:21586489
- 3. Steinvorth SM, Röver C, Schneider S, Nicholas R, Straube S, Friede T. Explaining temporal trends in annualized relapse rates in placebo groups of randomized controlled trials in relapsing multiple sclerosis: systematic review and meta-regression. Multiple Sclerosis Journal. 2013;19(12):1580–1586. pmid:23471144
- 4. Nicholas R, Straube S, Schmidli H, Pfeiffer S, Friede T. Time-patterns of annualized relapse rates in randomized placebo-controlled clinical trials in relapsing multiple sclerosis: A systematic review and meta-analysis. Multiple Sclerosis Journal. 2012;18(9):1290–1296. pmid:22422808
- 5. Stellmann JP, Neuhaus A, Herich L, Schippling S, Roeckel M, Daumer M, et al. Placebo cohorts in phase-3 MS treatment trials–predictors for on-trial disease activity 1990–2010 based on a meta-analysis and individual case data. PLoS ONE. 2012;7(11):e50347. pmid:23209717
- 6. Stellmann JP, Stürner KH, Young KL, Siemonsen S, Friede T, Heesen C. Regression to the mean and predictors of MRI disease activity in RRMS placebo cohorts–is there a place for baseline-to-treatment studies in MS? PLoS ONE. 2015;10(2):e0116559. pmid:25659100
- 7.
Committee for Medicinal Products for Human Use (CHMP). Guideline on clinical investigation of medicinal products for the treatment of Multiple Sclerosis. EMA/CHMP/771815/2011, Rev. 2, 2015.
- 8. Kurtzke JF. Rating neurologic impairment in multiple sclerosis. Neurology. 1983;33(11):1444–1452. pmid:6685237
- 9. Viechtbauer W. Conducting meta-analyses in R with the metafor package. Journal of Statistical Software. 2010;36(3):1–48.
- 10. Rukhin AL, Biggerstaff BJ, Vangel MG. Restricted maximum likelihood estimation of a common mean and the Mandel-Paule algorithm. Journal of Statistical Planning and Inference. 2000;83(2):319–330.
- 11. Kass AE, Raftery RE. Bayes Factors. Journal of the American Statistical Association. 1995;90(430):773–795.
- 12. Kicinski M, Springate DA Kontopantelis E. Publication bias in meta-analyses from the Cochrane Database of Systematic Reviews. Statistics in Medicine. 2015 (in press). [Epub ahead of print]
- 13. Aalen OO. Effects of frailty in survival analysis. Statistical Methods in Medical Research. 1994;3(3):227–234. pmid:7820293
- 14. Petkau J, Reingold SC, Held U, Cutter GR, Fleming TR, Hughes MD, et al. Magnetic resonance imaging as a surrogate outcome for multiple sclerosis relapses. Multiple Sclerosis. 2008;14(6):770–778. pmid:18535021
- 15. Rudick RA, Cutter G. MRI lesions: a surrogate for relapses in multiple sclerosis? The Lancet Neurology. 2013;12(7):628–630. pmid:23743083
- 16. Barkhof F. The clinico-radiological paradox in multiple sclerosis revisited. Current Opinion in Neurology. 2002;15(3):239–245. pmid:12045719
- 17. Thompson AJ, Hobart JC. Multiple sclerosis: assessment of disability and disability scales. Journal of Neurology. 1998;245(4):189–196. pmid:9591219
- 18. Hobart J, Kalkers N, Barkhof F, Uitdehaag B, Polman C, Thompson A. Outcome measures for multiple sclerosis clinical trials: relative measurement precision of the Expanded Disability Status Scale and Multiple Sclerosis Functional Composite. Multiple Sclerosis. 2004;10(1):41–46. pmid:14760951
- 19. Noseworthy JH, Vandervoort MK, Wong CJ, Ebers GC. Interrater variability with the Expanded Disability Status Scale (EDSS) and Functional Systems (FS) in a multiple sclerosis clinical trial. Neurology. 1990;40(6):971–975. pmid:2189084
- 20. Ontaneda D, Cohen JA. EDSS improvement: recovery of function or noise? Multiple Sclerosis Journal. 2012;18(11)1520–1521. pmid:23100522
- 21. Kontopantelis E, Springate DA, Reeves D. A re-analysis of the Cochrane library data: The dangers of unobserved heterogeneity in meta-analyses. PLoS ONE. 2013;8(7):e69930. pmid:23922860
- 22. Sormani MP, Rio J, Tintorè M, Signori A, Li D, Cornelisse P, et al. Scoring treatment response in patients with relapsing multiple sclerosis. Multiple Sclerosis Journal. 2013; 19(5):605–612. pmid:23012253
- 23. Hyun J-W, Kim S-H, Jeong IH, Ahn S-W, Huh S-Y, Park MS, et al. Utility of the Rio score and modified Rio score in Korean patients with multiple sclerosis. PLoS ONE. 2015; 10(5):e0129243. pmid:26010897