Multiple Sclerosis Journal - October 2017 - 1438

Multiple Sclerosis Journal 23(11)
it would have been useful to see baseline characteristics of the participants for whom data have been analysed versus those lost to follow-up.12
We are also unconvinced by the need to design three
RCTs rather one. The TREFAMS group argued that
three RCTs each recruiting from two separate centres
would minimise contamination across treatment
arms.4 While this may be logistically simpler, randomising 137 of 266 patients overall to control arms
with only between 42 and 44 people per trial randomised to active interventions is an inefficient
design. With one large trial, a greater proportion of
patients could have been randomised to active treatments providing greater power to detect treatment
effects across interventions. Furthermore, while
recruiting 266 patients to three studies is no small
task, we have concerns about the power of the studies
to detect important differences.
The power calculations for the three studies accounted
for 10% attrition rather than 20% attrition as stated in
the papers. Based on 20% attrition, sample sizes
should have been 50 per group rather than 45. A bigger
issue is that the studies were not powered to detect
appropriate effect sizes. The studies were powered to
detect an 8-point difference on the CIS20R (standardised mean difference (SMD) = 0.63), which while
being 'clinically significant' will likely be considerably larger than the minimum clinically important difference (MCID). We could find no published MCID
for the CIS20R. However, for other fatigue measures,
MCIDs are typically in the order of SMDs between 0.3
and 0.4.13 The studies were also not explicitly powered
to detect effects on the co-primary IPA outcome or any
of the secondary outcomes. IPA is a general measure
of quality of engagement in societal roles which can be
influenced by a wide range of general and MS-related
factors other than fatigue, and as such one would
expect smaller effects. Underpowered studies lead to a
higher risk of false-negative results and as such limit
inferences that can be drawn from the studies. Nonsignificant effects simply indicate that there is insufficient evidence to reject the null hypothesis, not that we
can infer no treatment effect exists.
Perhaps most importantly, without one large RCT, we
cannot draw direct inferences about the three treatment approaches. This is disappointing as to date, no
trials have explored the relative efficacy of these three
approaches. To have a better understanding of how
the findings across these three trials compare, we
have converted the treatment effects reported for the
CIS20R fatigue measure to SMDs and plotted these
out against time below (see Figure 1). Negative
1438

Figure 1. Standardised mean differences for each
intervention compared to control. Negative values favour
intervention.

values favour the intervention. All three interventions
show some effect in favour of the intervention at
16 weeks (but this is non-significant for the ECM
group). None are significant by 26 weeks, although
the effect size for CBT is still moderate. None of the
interventions show any effect after 1 year.
We have also used these SMDs to conduct a multivariate meta-analysis using a fixed-effect model,
which allows for the estimation of the uncertainty
around the differences in effect sizes across the three
studies.14 Bearing in mind this is indirect evidence,
we have made some cautious inferences with respect
to the different effects across studies. The effect size
for CBT compared to ECM is SMD = 0.57 (95% CI:
−0.05 to 1.19) in favour of CBT and compared to
exercise is SMD = 0.22 (95% CI: −0.38 to 0.82) in
favour of CBT. For exercise compared to ECM,
SMD = 0.35 (95% CI: −0.27 to 0.97). The confidence
intervals provide further evidence that compared to
CBT and aerobic exercise, ECM is unlikely to be efficacious. However, there is less certainty to draw firm
conclusions about CBT compared to aerobic exercise.
Obviously, there are just three studies here and a
proper network meta-analysis is needed, including all
published studies.
Although we cannot use this indirect evidence to draw
firm conclusions about the interventions studied, it
helps to inform the design of future RCTs in this area.
ECM appears to have a small effect size at best (further limited by dropout), so is not worth further
research on its own, particularly as there are now several EC studies showing small or null effects.15 Future
trials could use ECM as a better and more matched
control condition for either exercise or CBT. We also
need to focus on how to maintain treatment effects in
an illness where increasing disability, fluctuating
symptoms and relapse are likely. This may mean
journals.sagepub.com/home/msj

https://journals.sagepub.com/home/msj

Table of Contents for the Digital Edition of Multiple Sclerosis Journal - October 2017