Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Cutoff criteria for the placebo response: a cluster and machine learning analysis of placebo analgesia


Computations of placebo effects are essential in randomized controlled trials (RCTs) for separating the specific effects of treatments from unspecific effects associated with the therapeutic intervention. Thus, the identification of placebo responders is important for testing the efficacy of treatments and drugs. The present study uses data from an experimental study on placebo analgesia to suggest a statistical procedure to separate placebo responders from nonresponders and suggests cutoff values for when responses to placebo treatment are large enough to be separated from reported symptom changes in a no-treatment condition. Unsupervised cluster analysis was used to classify responders and nonresponders, and logistic regression implemented in machine learning was used to obtain cutoff values for placebo analgesic responses. The results showed that placebo responders can be statistically separated from nonresponders by cluster analysis and machine learning classification, and this procedure is potentially useful in other fields for the identification of responders to a treatment.


The use of placebos is crucial in medical science to determine the effects and efficacy of treatments1,2. Because randomized controlled trials (RCTs) use placebo conditions as a control, the definition of placebo responders is important to make valid statements about the statistical effects of treatments3. However, there is no consensus on how to statistically define a placebo responder or how to determine the cutoff for when a placebo response is large enough to be considered a true change from baseline. The majority of studies investigating the mechanistic components of the placebo response have used the placebo response in the context of pain, i.e., placebo analgesia, as the modality for studying this effect4.

Experimental studies on the mechanisms of the placebo response have traditionally relied on defining a placebo effect as the statistically significant difference in mean values between a group that receive placebo treatment and a group that receive no treatment but undergo the same assessments and procedures as the placebo group5. However, statistically significant differences are largely dependent on sample sizes, and larger samples might produce significant differences between groups that may have no practical or clinical relevance6. In the field of clinical pain, several authors have previously addressed this issue, and values for minimal clinically important differences in pain change have been suggested in terms of absolute changes and percentage changes. In a study on acute pain in trauma patients, Todd et al.7 found that an absolute pain reduction of 13 mm on a 100-mm visual analog scale (VAS) was a valid cutoff value for a clinically significant change in acute pain. The finding by Todd et al. was replicated in other studies8,9, but there was the notion that patients with higher pain at baseline required a higher VAS reduction compared to those with lower initial VAS ratings to achieve a meaningful reduction in pain9. The relative percentage change in pain might in some situations be less biased by baseline or pretreatment pain compared to the absolute VAS change, and Jensen et al.10 suggested that more than a 33% relative and 20–30 mm absolute reduction in individual VAS scores constituted a clinically meaningful change in postoperative pain. In a study employing meta-regression on data from 10,938 patients from 40 studies with various diagnoses of chronic pain, a minimum clinically important difference of 28 mm on the 100-mm VAS scale was found11. Thus, the cutoff values for a clinically relevant VAS change vary among acute, postoperative, and chronic types of pain.

However, most mechanistic studies on placebo analgesia have been performed in healthy volunteers who had no prior history of long-lasting pain. Thus, the most comparable clinical category is probably acute pain. By using the cutoff value from Todd et al.7, a meta-analysis on placebo effects12 found that only 26% of the included placebo studies performed in healthy volunteers reported clinically significant pain reduction after placebo treatment. Minor pain reductions in placebo analgesic studies could be problematic for the interpretation and generalizability of mechanistic studies on the placebo response. Furthermore, the reported response rate to placebo treatment varies substantially between studies and between experimental and clinical settings13,14. In an influential and debated study by Beecher15, the response rate to placebo in clinical pain was estimated to be 35.2%, whereas other more recent studies have shown values between 39–78%16,17 and a 56–72% response rate for experimental placebo analgesia18,19 when a responder was defined as a participant who reported a change in pain in the expected direction after placebo treatment. Hence, the majority of studies reporting the percentage of placebo responders define a placebo responder as a patient or a participant in an experimental study that reports any pain reduction measured on a VAS or numerical rating scale (NRS) after placebo treatment, regardless of the magnitude of the pain reduction12.

Here, data from a large experimental placebo analgesic study performed in healthy volunteers20 were used to test the assumption that placebo responders can be separated from nonresponders by cutoff values for both absolute and relative percentage changes in pain measured on a 100-mm VAS. Data from 296 participants (179 females) were included. Thermal pain stimulation was performed in two pretests and three posttests, and the temperature for pain stimulation needed to evoke a rating of 60 on a 100-mm computerized visual analog scale was individually determined for each participant in a calibration phase before the pretests. Placebo administration (placebo cream) together with information that the cream was a painkiller was performed after the pretests. The design consisted of a double-blind procedure in the placebo group, whereas the control group received no cream but experienced the same pain procedure as the placebo group.

Unsupervised cluster analysis was used to classify responders and nonresponders, and the classification was tested with different cutoffs in a supervised machine learning algorithm based on logistic regression. If a placebo analgesic manipulation produces either responders or nonresponders, an unsupervised cluster analysis should result in two clusters. Since placebo analgesia is dependent on a reduction in pain reports measured on the same scales as clinical pain, it was hypothesized that reductions in pain after placebo treatment should have the same magnitude as pain reduction after clinical treatment7,10,11 to refer to the response as “analgesia”. Consistent with the predictions, the cluster analysis revealed two distinct clusters in the data separated by significant differences in pain change after placebo treatment. Furthermore, the machine learning classification suggested cutoff values for placebo analgesic responses close to the suggested cutoffs for acute clinical pain. The assumption that pain reporting after placebo administration results in a two-cluster solution was supported by testing pain change data from three previously published studies from our lab21,22,23. Consequently, it is suggested that the statistical approach in the present study could be used as an alternative for defining placebo responders in both clinical and experimental studies where patient self-report is the main outcome.


Group differences

A comparison of pretest and posttest values in the placebo group showed a significant mean decrease in pain intensity of 16.76 VAS points (95% CI [13.94–19.59]) after placebo manipulations consisting of verbal information and application of a placebo cream (t (145) = 11.72, p < .001). The mean values of the two pretests were 58.24 (95% CI [55.67–61.11]), and the mean pain intensity in the posttest was 41.48 VAS points (95% CI [38.39–45.02]) on the 100-mm VAS scale. In contrast, in the control group that received no manipulations during the pain stimulation procedure, the mean decrease in pain was significant (t (124) = 3.04, p = .003), with a 3.26 VAS points reduction (95% CI [1.03–5.39]) from the pretest to the posttest. When comparing the placebo group and the control group, a significantly larger pain reduction was observed in the placebo group (t (294) =  − 7.85, p < .001). Hence, the design produced a statistically significant placebo analgesic effect.

Cluster analysis

A two-step unsupervised cluster analysis was performed on the VAS change scores for the mean of the two pretests and the last posttest (pretests–last posttest). Before performing the two-step cluster analysis, the order of the participants in the data file was randomized in order to reduce order-effects. In the placebo group, the analysis revealed two distinct clusters (Fig. 1) with means of 32.68 (95% CI [30.43–34.91]) and 5.16 (95% CI [2.98–7.33]), medians of 31 and 7.5, and modes of 29.5 and 3 for changes in VAS pain ratings. The cluster quality was good, with an average silhouette measure of .7. The two clusters were termed placebo responders (N = 65, 42%) and nonresponders (n = 90, 58%), respectively. The distribution of changes in pain for the responders and nonresponders is shown in Fig. 2, and the mean difference in pain changes between the responders and nonresponders was significant (t (153) = 17.19, p < .001). The effect size was very large24 based on the mean difference between the responders and the nonresponders (32.68–5.16/SD 16.70 = 1.65; Cohen's d). To test the stability of the results from the two-step cluster analysis, the order of the sample was randomized again and split into halves and then run in two separate samples. When splitting the sample, the results were identical to the initial analysis with two clusters and a silhouette measure of .7. The cluster solution with two clusters was additionally tested in a supervised k-means cluster analysis. The k-means analysis showed that a two-cluster solution produced the same number of responders (N = 65), and nonresponders (N = 90), the two clusters had final cluster centers of 32.68 and 4.88, and the ANOVA test for differences in cluster means was significant (F (1, 153) = 295.54, p < .001).

Figure 1
figure 1

Box and violin plots showing the density of placebo responders versus nonresponders.

Figure 2
figure 2

Histograms showing the distribution of responders and nonresponders classified by the two-step cluster analysis.

When performing the two-step cluster analytic procedure on data from the control group, four clusters emerged with good quality of the clusters (.7) determined by the average silhouette measure (cluster 1, N = 23, mean = 20.8; cluster 2, N = 42, mean = 8.83; cluster 3, N = 50, mean =  − .09; cluster 4, N = 26, mean =  − 13.13).

Validation of the two-cluster solution for placebo responding

The unsupervised cluster analysis for classification was additionally tested on data from the placebo arms and/or conditions in published studies with smaller samples from our lab21,22,23. The two-step cluster analysis classified the pain change data in two-cluster solutions in all three datasets, with silhouette measures .7, .7, and .6, respectively. Thus, the classification produced cluster solutions of good quality, and the assumption that self-reported responses after placebo administration can be classified in responders and non-responders was supported. See Supplemental information for details about the validation of the cluster solution.

Determination of cutoff values by machine learning

The two categories (placebo responders and nonresponders) derived from the cluster analysis in the placebo group were used as dependent variables in a machine learning algorithm based on logistic regression. Values for pain change ranging from 0 to 50 for percentage change and change values from 0 to 25 for absolute VAS change were tested as possible cutoff values. The false discovery rate (FDR) was used to assess the predictive ability for each of the selected cutoff values, and q ≤ 5% was chosen as the FDR criterion for a valid cutoff that could separate responders from nonresponders. The optimal cutoff values (Table 1) for a VAS change were 18–20. These values produced identical area under the curve (AUC) values (.99) and showed high accuracy (99.4%) for the prediction of the placebo responders identified by the cluster analysis. Furthermore, low (0–1%) FDR values for both false classification of responders and nonresponders suggested a highly precise prediction. A VAS change value of 17 also reached the FDR criterion but with somewhat lower AUC and accuracy values. In summary, absolute VAS changes in the range between 17 and 20 points could separate responders from nonresponders within the 5% FDR criterion. In comparison, a cutoff based on any or a small change in pain (≥ 0) showed a predictive value close to chance level (AUC = .61), whereas a cutoff based on a large VAS change score (≥ 25) correctly classified all the nonresponders but incorrectly classified 17% of the responders as nonresponders. Thus, using higher cutoff values may increase the false negative rate. When using different cutoffs for percentage change values (0 to 50% change in VAS ratings), the optimal cutoffs were 27–28% change (AUC = .99, accuracy = 99.3%) with FDR values of 0–1% (Table 1). Percentage changes between 25 and 30 all passed the 5% FDR criterion but with lower AUC (.96–.97) and accuracy values (97.3–97.9%). Calculation of the Youden index25 was additionally performed to compare cutoffs from the logistic machine learning analysis with a widely used method for this purpose26. Generally, the machine learning procedure and the Youden method gave similar cutoff values (see Table 1).

Table 1 Cutoff values from logistic regression machine learning.


The gold standard for the determination of a placebo effect in experimental designs is the observation of statistically significant differences between one or more groups receiving placebo treatment and a group receiving the same procedure but with no treatment27. Hence, this definition is based on observed differences in measures of central tendencies, most often mean differences, regardless of the magnitude of the change; when statistical significance is used as the only criterion, this might not account for the physiological and psychological experience of the stimulation or whether analgesia is induced.

The results from the present study and studies on cutoffs for clinical pain7,8,9,10,11 suggest that small, albeit statistically significant, changes in pain after treatment may have no practical or clinical relevance. Furthermore, when using any pain change in the desired direction as the definition of placebo responders, the present data showed that the accuracy was close to chance. This is a problem for the interpretation of experimental studies on placebo analgesia with no criterion for defining when changes in pain magnitude are sufficient to be considered meaningful decreases in pain perception. However, studies with small but significant changes in pain may still provide important information about mechanisms of placebo-induced changes in physiological systems related to pain perception21,28 and how contextual and psychological factors associated with placebo administration may affect treatment or experimental outcomes; see2,29,30 for an overview. Nonetheless, the term “analgesia” refers to the relief of pain without the loss of consciousness, and studies making inferences about placebo analgesia should at least display changes in pain ratings large enough to be recognized as reduced pain intensity that are perceived as such.

The results from the present study showed a significant difference in mean pain change between the placebo group and the control group. Nonetheless, the cluster analysis revealed that one of the clusters in the control group data had a pain reduction that could be classified as placebo responses according to the cutoff values found in the present study if it had been observed in the placebo group. Moreover, the second cluster in the control group had a mean pain change of 8.83, which is comparable to placebo induced pain changes in several previous studies5,12,31,32. These findings suggests that relatively large changes in experimental pain reports could occur in the absence of a treatment and be caused by factors such as reporting biases, individual differences in emotional responses to the pain stimulation, and statistical regression to the mean33,34,35. Thus, the possibility exist that parts of the improvement observed in the placebo group would also have occurred without the placebo treatment36.

The present study is probably the first to provide cutoff values for experimental placebo analgesia. The optimal cutoff values of 18–20 mm for absolute VAS changes and 27–28% changes are not surprising given findings in studies on clinical pain7,8,9,10,11. However, the cutoff suggested by Todd et al.7 of a 13-mm change for acute pain was too liberal in the present data, as shown by both the machine learning analysis and the Youden index.

Nonetheless, the cutoffs found in the present study were based on a specific procedure for placebo treatment and pain stimulation, and different procedures might provide different values. A limitation of the present data is that the pain measurement did not include a qualitative measure for rating the perceived pain37 after placebo treatment as “much less", "little less", "the same", etc. Hence, minimally clinically important difference (MCID) values could not be computed. On the other hand, the present results are in line with suggested cutoff values found in studies on clinical pain, and the pain stimulation used here is one of the most common methods in experimental pain studies38. The use of cutoff values for pain changes to perform responder analyses has been recommended for clinical pain trials with the reasoning that statistical mean differences might not capture the experienced effect of pain reductions in most patients39,40,41. Thus, the definition of placebo responders versus nonresponders based on cutoffs could be a beneficial add-on for analyses for both clinical and experimental studies. However, the terms “placebo responders and nonresponders” does not imply that participants who shows a positive response to placebo administration in e.g., a pain study will be placebo responders in other situations. Placebo responding and placebo nonresponding are probably not consistent traits14,42, but depend on the individual learning history of the participants43, level of emotional activation during treatment44, and genetic factors20,45 associated with the sensory modality in which the placebo treatment is used.

The magnitude and impact of placebo analgesic effects in clinical trials and experimental studies have been debated, e.g.,12,31,46,47. Nonetheless, these meta-analyses have shown that placebo administration has a statistical impact on self-reported pain. Furthermore, Zunhammer et al.48,49 found in meta-analyses that placebo administration has minor effects on the neurologic pain signature50 but moderate effects on pain reports, which suggests that the measurement and analysis of pain reports are crucial for determining the magnitude of placebo analgesic effects. Nevertheless, most meta-analyses have not analyzed whether the observed effects of placebos had a clinically meaningful impact. To date, the only meta-analysis that tested clinical significance in relation to placebo analgesia found that the placebo analgesic effect was higher in patients than in healthy controls in terms of clinically significant reductions in pain when using the reduction criteria of ≥ 13 mm7 on a 0–100 VAS or NRS scale12. Interestingly, meta-analyses that have used effect sizes as the main outcome measure have shown the opposite finding: the placebo effect is larger in healthy controls than in patients31,32. Taken together, analyses of placebo effects in future studies should preferably include assessments of both statistical and clinical significance, at least in studies making inferences about patient-reported outcomes such as pain51.

The present study shows that placebo responders can be separated from nonresponders with a straightforward procedure based on unsupervised cluster analysis and cutoffs selected by logistic regression implemented in machine learning. These statistical tools are available in several statistical software packages and could easily be used by scientists with some experience in statistics. Moreover, the machine learning application used here can be replaced by an ROC curve analysis providing data for sensitivity and specificity52 that is implemented in most statistical programs or software specialized for finding optimal cutoff points53. However, the latter does require that the patients/participants be classified as either responders or nonresponders before the analysis to find the optimal cutoff is performed. Thus, in cases where there is no diagnostic information or clinical classification, an unsupervised cluster analysis may help the categorization process without inducing bias associated with manual human classification. The two-cluster classification of the placebo responses of the present study was replicated in other data from our lab. However, other types of experimental designs may provide more complex classification of responses54. In data where more than two clusters are found, the logistic regression and eventually the Youden index must be replaced with classification analyses that handle multiclass data such as support vector machines or similar methods. An advantage of machine learning algorithms for classification over standard statistical methods is that these applications provide additional statistics, such as FDR values, compared to other analysis such as logistic regression. Furthermore, the flexibility in model specifications, such as validation schemes of the selected model, provides better control over the accuracy and error rate compared to standard statistical methods.


Data from 296 healthy subjects who participated in an experimental study on genetic factors in placebo analgesia20 were used for the present analyses. The characteristics of the sample are described in detail in previous publications20,55.

The participants were randomized into three groups: (1) a placebo group (N = 155) that received a moisturizing cream with no analgesic properties (E-45; Crookes Healthcare, Nottingham, United Kingdom), (2) a natural history group (N = 141) that received no treatment during the procedure, or (3) a lidocaine-prilocaine cream group (N = 31) that received a local anesthetic cream (Emla; AstraZeneca, Oslo, Norway)20. Thus, the study included a total of 327 healthy participants with a mean age of 23 years (SD = 3.3). A dose of 3 g of Emla or E-45 was used for each participant. The group receiving the anesthetic cream was used in the design to ensure blinding of the experimenters, and these data were not used in the analyses20,55. The experiment was run in a double-blind manner for the groups where a cream was applied, but there was no blinding in the natural history group. Randomization was performed before the start of the experiment. The participants were allocated to the different groups based on their participant number. The participant numbers and group allocations were randomized by using the online web service Thirty-one of the participants were randomized into the lidocaine-prilocaine cream group. Hence, data from 296 participants were included in the final analyses20. The study was reviewed and approved by the Regional Committee for Medical and Health Research Ethics, Region Northern Norway (project number 23430), and the study was performed in accordance with The Declaration of Helsinki. All participants signed an informed consent before participation. In the consent form, they stated that they had no history of ongoing disease or any history of serious disease.

Design and stimuli

The study was designed as a repeated measures design with a calibration phase for determination of temperatures that invoked pain corresponding to 60 mm on a 100-mm computerized visual analog scale (VAS) scale. Thermal stimuli to the left underarm were delivered by a computer-controlled thermode (Pathway, Medoc, Israel) to induce heat pain. To ensure an equal pain level across participants at the start of the experiment, a calibration procedure was performed. To approximate the stimulus intensity needed to produce a rating of 60 on the VAS, Stevens power equation56 was used to produce a stimulus–response function for estimation of the individual target temperature of the thermode. After the calibration phase, the pretest consisting of the presentation of two pain stimuli was performed. The duration of the stimulations in the pretests and posttests was 10 s from when the thermode reached the calibrated target temperature (43–47 °C) until the start of the return to baseline at 32 °C20,55. The temperature of the thermode increased/decreased by 10 °C/s. The interval between the two pretests was 30 s. The two posttests had the same temperature, duration, and intervals as the pretests20,55. Directly after the pretests, information about the treatment was provided to the participants allocated to the groups that received either placebo or Emla20,55. The participants in the placebo group were told “The cream that will be applied to your arm reduces pain, the substance in the cream is used as a local anesthetic in many pain-reducing remedies and is effective in the treatment of heat pain.” The participants in the natural history group received no application of cream and were told that they could relax in the break between the pretests and the posttests20,55. The experimental procedure had a total duration of approximately 45 min, which included saliva sampling for genetic analyses and measurements of blood pressure and subjective stress.

Statistical analyses

A two-step cluster analysis in SPSS v. 26 (SPSS, IBM, USA) was used to classify changes in VAS pain intensity from the pretest to the posttest. The order of the participants in the datafile was randomized by the randperm function in MATLAB (MATLAB v.R2019b, Sweden). The cluster analysis was set to automatically determine the optimal number of clusters by using Schwarz’s Bayesian criterion57 and log-likelihood as the distance measure . The quality of the cluster analysis was measured by the silhouette coefficient, which is a measure of both cohesion and separation of the suggested clusters. MATLAB Classification Learner (MATLAB, Sweden; with cross-validation (fivefolds) was used for the prediction of cutoff values from the two-step cluster analysis. The logistic regression classifier was used for classification of the selected possible cutoffs for the absolute VAS change and percentage change between the pre- and posttests. The possible cutoffs were tested separately. Predictive values were determined from ROC curves providing area under the curve (AUC) values and a confusion matrix that provided false discovery rate (FDR) values. The accuracy of the predictor is in the Classification Learner calculated from the cross-validation. The Youden index was calculated in SPSS based on a script ( Group comparisons were performed with two-tailed t-tests. p values < .05 were considered significant.

Data availability

All data used in the analyses for this article can be found at


  1. Colloca, L. & Barsky, A. J. Placebo and Nocebo effects. N. Engl. J. Med. 382, 554–561. (2020).

    CAS  Article  PubMed  Google Scholar 

  2. Wager, T. D. & Atlas, L. Y. The neuroscience of placebo effects: Connecting context, learning and health. Nat. Rev. Neurosci. 16, 403–418. (2015).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  3. Coleshill, M. J., Sharpe, L., Colloca, L., Zachariae, R. & Colagiuri, B. International Review of Neurobiology Vol. 139, 407–441 (Elsevier, 2018).

    Google Scholar 

  4. Finniss, D. G., Kaptchuk, T. J., Miller, F. & Benedetti, F. Biological, clinical, and ethical advances of placebo effects. Lancet 375, 686–695. (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Vase, L. & Petersen, G. L. Placebo and Pain 203–213 (Elsevier, 2013).

    Book  Google Scholar 

  6. Angst, F., Aeschlimann, A. & Angst, J. The minimal clinically important difference raised the significance of outcome effects above the statistical level, with methodological implications for future studies. J. Clin. Epidemiol. 82, 128–136. (2017).

    Article  PubMed  Google Scholar 

  7. Todd, K. H., Funk, K. G., Funk, J. P. & Bonacci, R. Clinical significance of reported changes in pain severity. Ann. Emerg. Med. 27, 485–489. (1996).

    CAS  Article  PubMed  Google Scholar 

  8. Gallagher, E. J., Liebman, M. & Bijur, P. E. Prospective validation of clinically important changes in pain severity measured on a visual analog scale. Ann. Emerg. Med. 38, 633–638. (2001).

    CAS  Article  PubMed  Google Scholar 

  9. Bird, S. B. & Dickson, E. W. Clinically significant changes in pain along the visual analog scale. Ann. Emerg. Med. 38, 639–643. (2001).

    CAS  Article  PubMed  Google Scholar 

  10. Jensen, M. P., Chen, C. & Brugger, A. M. Interpretation of visual analog scale ratings and change scores: A reanalysis of two clinical trials of postoperative pain. J. Pain 4, 407–414 (2003).

    Article  Google Scholar 

  11. Olsen, M. F. et al. Minimum clinically important differences in chronic pain vary considerably by baseline pain and methodological factors: Systematic review of empirical studies. J. Clin. Epidemiol. 101, 87-106.e102. (2018).

    Article  PubMed  Google Scholar 

  12. Forsberg, J. T., Martinussen, M. & Flaten, M. A. The placebo analgesic effect in healthy individuals and patients: A meta-analysis. Psychosom. Med. 79, 388–394. (2017).

    Article  PubMed  Google Scholar 

  13. Enck, P., Klosterhalfen, S., Weimer, K., Horing, B. & Zipfel, S. The placebo response in clinical trials: More questions than answers. Philos. Trans. R. Soc. B 366, 1889–1895. (2011).

    Article  Google Scholar 

  14. Kaptchuk, T. J. et al. Do “placebo responders” exist?. Contemp. Clin. Trials 29, 587–595 (2008).

    Article  Google Scholar 

  15. Beecher, H. K. The powerful placebo. J. Am. Med. Assoc. 159, 1602–1606. (1955).

    CAS  Article  PubMed  Google Scholar 

  16. Finniss, D., Nicholas, M., Brooker, C., Cousins, M. & Benedetti, F. Magnitude, response, and psychological determinants of placebo effects in chronic low-back pain: A randomised, double-blinded, controlled trial. Pain Rep. 4, e744. (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  17. Levine, J. D., Gordon, N. C., Bornstein, J. C. & Fields, H. L. Role of pain in placebo analgesia. Proc. Natl. Acad. Sci. U. S. A. 76, 3528–3531. (1979).

    ADS  CAS  Article  PubMed  PubMed Central  Google Scholar 

  18. Petrovic, P., Kalso, E., Petersson, K. M. & Ingvar, M. Placebo and opioid analgesia—Imaging a shared neuronal network. Science 295, 1737–1740. (2002).

    ADS  CAS  Article  PubMed  Google Scholar 

  19. Wager, T. D. et al. Placebo-induced changes in FMRI in the anticipation and experience of pain. Science 303, 1162–1167. (2004).

    ADS  CAS  Article  PubMed  Google Scholar 

  20. Aslaksen, P. M., Forsberg, J. T. & Gjerstad, J. The opioid receptor mu 1 (OPRM1) rs1799971 and catechol-O-methyltransferase (COMT) rs4680 as genetic markers for placebo analgesia. Pain 159, 2585–2592. (2018).

    CAS  Article  PubMed  Google Scholar 

  21. Aslaksen, P. M. & Flaten, M. A. The roles of physiological and subjective stress in the effectiveness of a placebo on experimentally induced pain. Psychosom. Med. 70, 811–818. (2008).

    Article  PubMed  Google Scholar 

  22. Aslaksen, P. M., Vasylenko, O. & Fagerlund, A. J. The effect of transcranial direct current stimulation on experimentally induced heat pain. Exp. Brain Res. 232, 1865–1873. (2014).

    Article  PubMed  Google Scholar 

  23. Aslaksen, P. M., Zwarg, M. L., Eilertsen, H. H., Gorecka, M. M. & Bjorkedal, E. Opposite effects of the same drug: reversal of topical analgesia by nocebo information. Pain 156, 39–46. (2015).

    Article  PubMed  Google Scholar 

  24. Sawilowsky, S. S. New effect size rules of thumb. J. Mod. Appl. Stat. Methods 8, 26 (2009).

    Article  Google Scholar 

  25. Youden, W. J. Index for rating diagnostic tests. Cancer 3, 32–35.<32::aid-cncr2820030106>;2-3 (1950).

    CAS  Article  PubMed  Google Scholar 

  26. Fluss, R., Faraggi, D. & Reiser, B. Estimation of the Youden Index and its associated cutoff point. Biometr. J. 47, 458–472. (2005).

    MathSciNet  Article  MATH  Google Scholar 

  27. Linde, K., Fassler, M. & Meissner, K. Placebo interventions, placebo effects and clinical practice. Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci. 366, 1905–1912. (2011).

    Article  Google Scholar 

  28. Johansen, O., Brox, J. & Flaten, M. A. Placebo and Nocebo responses, cortisol, and circulating beta-endorphin. Psychosom. Med. 65, 786–790. (2003).

    CAS  Article  PubMed  Google Scholar 

  29. Benedetti, F., Carlino, E. & Pollo, A. How placebos change the patient’s brain. Neuropsychopharmacology 36, 339–354. (2011).

    Article  PubMed  Google Scholar 

  30. Enck, P., Bingel, U., Schedlowski, M. & Rief, W. The placebo response in medicine: Minimize, maximize or personalize?. Nat. Rev. Drug Discov. 12, 191–204. (2013).

    CAS  Article  PubMed  Google Scholar 

  31. Vase, L., Petersen, G. L., Riley, J. L. 3rd. & Price, D. D. Factors contributing to large analgesic effects in placebo mechanism studies conducted between 2002 and 2007. Pain 145, 36–44. (2009).

    Article  PubMed  Google Scholar 

  32. Vase, L., Riley, J. L. & Price, D. D. A comparison of placebo effects in clinical analgesic trials versus studies of placebo analgesia. Pain 99, 443–452 (2002).

    Article  Google Scholar 

  33. Bjorkedal, E. & Flaten, M. A. Interaction between expectancies and drug effects: An experimental investigation of placebo analgesia with caffeine as an active placebo. Psychopharmacology 215, 537–548. (2011).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  34. Geers, A. L. et al. Affect and emotions in placebo and nocebo effects: What do we know so far?. Soc. Pers. Psychol. Compass 15, e12575 (2021).

    Article  Google Scholar 

  35. Hróbjartsson, A. What are the main methodological problems in the estimation of placebo effects?. J. Clin. Epidemiol. 55, 430–435 (2002).

    Article  Google Scholar 

  36. Krogsbøll, L. T., Hróbjartsson, A. & Gøtzsche, P. C. Spontaneous improvement in randomised clinical trials: Meta-analysis of three-armed trials comparing no treatment, placebo and active intervention. BMC Med. Res. Methodol. 9, 1–10 (2009).

    Article  Google Scholar 

  37. Farrar, J. T., Portenoy, R. K., Berlin, J. A., Kinman, J. L. & Strom, B. L. Defining the clinically important difference in pain outcome measures. Pain 88, 287–294. (2000).

    Article  PubMed  Google Scholar 

  38. Baumgartner, U., Cruccu, G., Iannetti, G. D. & Treede, R. D. Laser guns and hot plates. Pain 116, 1–3.|ISSN0304-3959 (2005).

    Article  PubMed  Google Scholar 

  39. Woodcock, J., Witter, J. & Dionne, R. A. Stimulating the development of mechanism-based, individualized pain therapies. Nat. Rev. Drug Discov. 6, 703–710. (2007).

    CAS  Article  PubMed  Google Scholar 

  40. Dionne, R. A., Bartoshuk, L., Mogil, J. & Witter, J. Individual responder analyses for pain: Does one pain scale fit all?. Trends Pharmacol. Sci. 26, 125–130 (2005).

    CAS  Article  Google Scholar 

  41. Dworkin, R. H. et al. Research design considerations for confirmatory chronic pain clinical trials: IMMPACT recommendations. PAIN® 149, 177–193 (2010).

    Article  Google Scholar 

  42. Enck, P., Klosterhalfen, S. & Weimer, K. Unsolved, forgotten, and ignored features of the placebo response in medicine. Clin. Ther. 39, 458–468. (2017).

    Article  PubMed  Google Scholar 

  43. Muller, M. et al. Treatment history and placebo responses to experimental and clinical pain in chronic pain patients. Eur. J. Pain 20, 1530–1541. (2016).

    CAS  Article  PubMed  Google Scholar 

  44. Flaten, M. A., Aslaksen, P. M., Lyby, P. S. & Bjorkedal, E. The relation of emotions to placebo responses. Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci. 366, 1818–1827. (2011).

    Article  Google Scholar 

  45. Colloca, L. et al. OPRM1 rs1799971-COMT rs4680-FAAH rs324420 genes interact with placebo procedures to induce hypoalgesia. Pain 160, 1824 (2019).

    CAS  Article  Google Scholar 

  46. Hrobjartsson, A. & Gotzsche, P. C. Placebo interventions for all clinical conditions. Cochrane Database Syst. Rev. 2010, CD003974. (2010).

    Article  PubMed Central  Google Scholar 

  47. Hrobjartsson, A. & Gotzsche, P. C. Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. N. Engl. J. Med. 344, 1594–1602 (2001).

    CAS  Article  Google Scholar 

  48. Zunhammer, M., Bingel, U., Wager, T. D. & Placebo Imaging, C. Placebo effects on the neurologic pain signature: A meta-analysis of individual participant functional magnetic resonance imaging data. JAMA Neurol. 75, 1321–1330. (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  49. Zunhammer, M., Spisak, T., Wager, T. D., Bingel, U. & Placebo Imaging, C. Meta-analysis of neural systems underlying placebo analgesia from individual participant fMRI data. Nat. Commun. 12, 1391. (2021).

    ADS  CAS  Article  PubMed  PubMed Central  Google Scholar 

  50. Wager, T. D. et al. An fMRI-based neurologic signature of physical pain. N. Engl. J. Med. 368, 1388–1397. (2013).

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  51. Muñoz-Leyva, F., El-Boghdadly, K. & Chan, V. Is the minimal clinically important difference (MCID) in acute pain a good measure of analgesic efficacy in regional anesthesia?. Reg. Anesth. Pain Med. 45, 1000–1005 (2020).

    Article  Google Scholar 

  52. Hajian-Tilaki, K. The choice of methods in determining the optimal cut-off value for quantitative diagnostic test evaluation. Stat. Methods Med. Res. 27, 2374–2383. (2018).

    MathSciNet  Article  PubMed  Google Scholar 

  53. López-Ratón, M., Rodríguez-Álvarez, M. X., Cadarso-Suárez, C. & Gude-Sampedro, F. OptimalCutpoints: An R package for selecting optimal cutpoints in diagnostic tests. J. Stat. Softw. 61, 1–36 (2014).

    Article  Google Scholar 

  54. Mbowe, O. B., Gewandter, J. S., Turk, D. C., Dworkin, R. H. & McDermott, M. P. Are there really only 2 kinds of people in the world? Evaluating the distribution of change from baseline in pain clinical trials. Pain 161, 195–201 (2020).

    Article  Google Scholar 

  55. Forsberg, J. T., Gjerstad, J., Flaten, M. A. & Aslaksen, P. M. Influence of catechol-O-methyltransferase Val158Met on fear of pain and placebo analgesia. Pain 159, 168–174. (2018).

    CAS  Article  PubMed  Google Scholar 

  56. Stevens, S. S. To honor Fechner and repeal his law: A power function, not a log function, describes the operating characteristic of a sensory system. Science 133, 80–86. (1961).

    ADS  CAS  Article  PubMed  Google Scholar 

  57. Schwarz, G. Estimating the dimension of a model. Ann. Stat. 6, 461–464 (1978).

    MathSciNet  Article  Google Scholar 

Download references


The author thanks Espen Bjørkedal, PhD, for valuable comments in the process of writing the article.

Author information

Authors and Affiliations



P.M.A. was responsible for all aspects of the present study.

Corresponding author

Correspondence to Per M. Aslaksen.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Aslaksen, P.M. Cutoff criteria for the placebo response: a cluster and machine learning analysis of placebo analgesia. Sci Rep 11, 19205 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI:


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing