Most of our readers are aware of the distinction between statistical and clinical significance, and the importance of looking beyond p values when interpreting results of clinical trials (and for that matter the results of any type of research). However, clinical significance is not easily quantified . It relies on defining a minimally important treatment effect; a concept which researchers have grappled with since the 1980s . A paper published in Spinal Cord in 2015 provides a comprehensive summary on this topic . So the purpose of this editorial is primarily to direct readers back to this paper and provide a bite-size summary of some of the key concepts. This editorial will also hopefully serve as a reminder that there is no single value for an outcome that can be used across different trials to reflect a minimally important treatment effect.
There are many similar terms used to refer to a minimally important treatment effect. These include:
minimally important difference
smallest clinically significant effect
minimal clinically important difference
minimally worthwhile treatment effect
(see here for a summary of terminology ).
Some have argued against the use of the words, smallest or minimal, because treatments worth administering should have effects greater than implied in these two words. Hence, some have advocated for the term: sufficiently important difference . The use of the word “sufficient” is helpful because it begs for clarification. So sufficient for what? The answer is sufficient:
Regardless of terminology, a minimally (or sufficiently) important treatment effect is difficult to define because it is very context specific and requires consideration of the priorities of patients, the likelihood and implications of harm, and the ability of patients, funders or societies to pay. For example, patients may be prepared to engage in extremely strenuous physical training for 6 months to regain even the slightest improvement in mobility even if the training takes them away from their families, work and other purposeful life pursuits. However they may not be prepared to undergo this type of training if it is associated with substantial risk of harm. The cost of an intervention is also clearly an important consideration. A costly intervention with only small benefits but little risk of harm may be quite reasonable in a wealthy society but not reasonable in a low or middle-income country where the healthcare dollar is scarce and there are many competing life-and-death demands for resources.
A minimally important treatment effect is not the same as a minimal detectable difference, even though the two are commonly conflated. A minimal detectable difference is a psychometric property of an outcome measurement. It refers to the amount of change over time in an outcome measurement that needs to be seen in an individual to be confident that the change reflects more than measurement error. It does not provide insights into the amount of change needed to justify the time, cost and risk of harm of an intervention. Similarly, no statistic that relies on the distribution of a variable can be sensibly used to capture a minimally important treatment effect. Hence statistics such as effect sizes, standardised response means or a response statistic are not helpful in this context [4, 6] even though they continue to be widely use.
It is also rarely appropriate to imply that a specific outcome has a unique value which can then be used in different trials to reflect a minimally important treatment effect because the value depends on the intervention as well as the comparison of interest. For example, just because one needs to walk at 1.2 m.s−1 to safely cross a road does not mean that this value can be used as a minimally important treatment effect in every trial with walking speed as an outcome. This value might be too low in trials of very expensive and potentially harmful interventions but too high in trials involving interventions associated with minimal time, effort and cost. Importantly, the minimally important treatment effect needs to reflect an effect, not an outcome. An effect is the difference in outcome between the control and experimental conditions . In some trials the control condition will be no treatment but more commonly the control condition will be an alternate treatment. If a trial examines the relative effectiveness of two similar treatments with similar cost and risk of harm, then it may be quite reasonable to set the minimally important treatment effect quite low.
A minimally important treatment effect is very context specific and cannot be easily attained from the literature. The exception is studies which have explicitly set out to determine the minimally important treatment effect for a particular intervention compared to some clearly stated comparator. These types of studies are rare in spinal cord injuries although potentially very valuable. They require gauging patients’ preferences for particular interventions using quite sophisticated methodologies including benefit-harm trade-off methodologies . The participants must be reflective of those likely to receive the treatment.
In the absence of good quality data which defines a minimally important treatment effect, how should trialists define this value? At the very least trialists should informally consult with colleagues and patients. They need to pose the following question:
Importantly, a minimally important treatment effect needs to be defined as a between-group difference for at least the primary outcome before the start of a clinical trial (or any other type of study designed to determine treatment effectiveness). This needs to be documented in the protocol and the trial registry. Researchers then need to interpret the final results of their trial with respect to this value. Readers can disagree with the value and interpret the results accordingly to suit their context. However, upon completion of a trial, researchers should not redefine the value of their minimally important treatment effect.
Spinal Cord encourages all to make a clear distinction between minimally important treatment effects and minimal detectable differences. It also reminds its authors and readers that minimally important treatment effects need to be expressed as effects which capture the difference between the control and experimental conditions. Spinal Cord welcomes research aimed at quantifying the minimally important treatment effects for specific interventions provided it is appreciated that these values are context specific.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.