Article | Published:

# Efficient coding of subjective value

## Abstract

Preference-based decisions are essential for survival, for instance, when deciding what we should (not) eat. Despite their importance, preference-based decisions are surprisingly variable and can appear irrational in ways that have defied mechanistic explanations. Here we propose that subjective valuation results from an inference process that accounts for the structure of values in the environment and that maximizes information in value representations in line with demands imposed by limited coding resources. A model of this inference process explains the variability in both subjective value reports and preference-based choices, and predicts a new preference illusion that we validate with empirical data. Interestingly, the same model explains the level of confidence associated with these reports. Our results imply that preference-based decisions reflect information-maximizing transmission and statistically optimal decoding of subjective values by a limited-capacity system. These findings provide a unified account of how humans perceive and valuate the environment to optimally guide behavior.

## Access optionsAccess options

from\$8.99

All prices are NET prices.

## Data availability

The data that support the findings of this study and the analysis code are available from the corresponding author upon reasonable request.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## References

1. 1.

Barlow, H. B. in Sensory Communication (ed. Rosenblith, W. A.) 217–234 (MIT Press, Boston, 1961).

2. 2.

Attneave, F. Some informational aspects of visual perception. Psychol. Rev. 61, 183–193 (1954).

3. 3.

Ganguli, D. & Simoncelli, E. P. Efficient sensory encoding and Bayesian inference with heterogeneous neural populations. Neural Comput. 26, 2103–2134 (2014).

4. 4.

Wei, X.-X. & Stocker, A. A. Lawful relation between perceptual bias and discriminability. Proc. Natl Acad. Sci. USA 114, 10244–10249 (2017).

5. 5.

Laughlin, S. A simple coding procedure enhances a neuron’s information capacity. Z. Naturforsch. C 36, 910–912 (1981).

6. 6.

Stocker, A. A. & Simoncelli, E. P. Noise characteristics and prior expectations in human visual speed perception. Nat. Neurosci. 9, 578–585 (2006).

7. 7.

Körding, K. P. & Wolpert, D. M. Bayesian integration in sensorimotor learning. Nature 427, 244–247 (2004).

8. 8.

Petzschner, F. H., Glasauer, S. & Stephan, K. E. A Bayesian perspective on magnitude estimation. Trends. Cogn. Sci. 19, 285–293 (2015).

9. 9.

Wei, X.-X. & Stocker, A. A. A Bayesian observer model constrained by efficient coding can explain ‘anti-Bayesian’ percepts. Nat. Neurosci. 18, 1509–1517 (2015).

10. 10.

Lichtenstein, S. & Slovic, P. The Construction of Preference (Cambridge Univ. Press, Cambridge, 2006).

11. 11.

Polanía, R., Krajbich, I., Grueschow, M. & Ruff, C. C. Neural oscillations and synchronization differentially support evidence accumulation in perceptual and value-based decision making. Neuron 82, 709–720 (2014).

12. 12.

Krajbich, I., Armel, C. & Rangel, A. Visual fixations and the computation and comparison of value in simple choice. Nat. Neurosci. 13, 1292–1298 (2010).

13. 13.

Hunt, L. T. et al. Mechanisms underlying cortical activity during value-guided choice. Nat. Neurosci. 15, 470–476 (2012).

14. 14.

De Martino, B., Fleming, S. M., Garrett, N. & Dolan, R. J. Confidence in value-based choice. Nat. Neurosci. 16, 105–110 (2013).

15. 15.

Lebreton, M., Abitbol, R., Daunizeau, J. & Pessiglione, M. Automatic integration of confidence in the brain valuation signal. Nat. Neurosci. 18, 1159–1167 (2015).

16. 16.

Summerfield, C. & Tsetsos, K. Do humans make good decisions? Trends. Cogn. Sci. 19, 27–34 (2015).

17. 17.

Louie, K. & Glimcher, P. W. Efficient coding and the neural representation of value. Ann. N. Y. Acad. Sci. 1251, 13–32 (2012).

18. 18.

Glimscher, P. W., Camerer, C., Fehr, E. & Poldrack, A. Neuroeconomics: Decision Making and the Brain (Elsevier, Amsterdam, The Netherlands, 2008).

19. 19.

Woodford, M. Prospect theory as efficient perceptual distortion. Am. Econ. Rev. 102, 41–46 (2012).

20. 20.

Summerfield, C. & Tsetsos, K. Building bridges between perceptual and economic decision-making: neural and computational mechanisms. Front. Neurosci. 6, 70 (2012).

21. 21.

Khaw, M. W., Li, Z. & Woodford, M. Cognitive imprecision and small-stakes RISK aversion. NBER Working Paper No. 23294 https://doi.org/10.3386/w24978 (2018).

22. 22.

Padoa-Schioppa, C. Range-adapting representation of economic value in the orbitofrontal cortex. J. Neurosci. 29, 14004–14014 (2009).

23. 23.

Rustichini, A., Conen, K. E., Cai, X. & Padoa-Schioppa, C. Optimal coding and neuronal adaptation in economic decisions. Nat. Commun. 8, 1208 (2017).

24. 24.

Cox, K. M. & Kable, J. W. BOLD subjective value signals exhibit robust range adaptation. J. Neurosci. 34, 16533–16543 (2014).

25. 25.

Khaw, M. W., Glimcher, P. W. & Louie, K. Normalized value coding explains dynamic adaptation in the human valuation process. Proc. Natl Acad. Sci. USA 114, 12696–12701 (2017).

26. 26.

Girshick, A. R., Landy, M. S. & Simoncelli, E. P. Cardinal rules: visual orientation perception reflects knowledge of environmental statistics. Nat. Neurosci. 14, 926–932 (2011).

27. 27.

Louie, K., Khaw, M. W. & Glimcher, P. W. Normalization is a general neural mechanism for context-dependent decision making. Proc. Natl Acad. Sci. USA 110, 6139–6144 (2013).

28. 28.

Grueschow, M., Polania, R., Hare, T. A. & Ruff, C. C. Automatic versus choice-dependent value representations in the human brain. Neuron 85, 874–885 (2015).

29. 29.

Hare, T. A., Schultz, W., Camerer, C. F., O’Doherty, J. P. & Rangel, A. Transformation of stimulus value signals into motor commands during simple choice. Proc. Natl Acad. Sci. USA 108, 18120–18125 (2011).

30. 30.

Shadlen, M. N. N. & Shohamy, D. Decision making and sequential sampling from memory. Neuron 90, 927–939 (2016).

31. 31.

Milosavljevic, M., Malmaud, J., Huth, A., Koch, C. & Rangel, A. The drift diffusion model can account for value-based choice response times under high and low time pressure. Judgem. Decis. Mak. 5, 437–449 (2010).

32. 32.

Gluth, S., Sommer, T., Rieskamp, J. & Büchel, C. Effective connectivity between hippocampus and ventromedial prefrontal cortex controls preferential choices from memory. Neuron 86, 1078–1090 (2015).

33. 33.

Pouget, A., Drugowitsch, J. & Kepecs, A. Confidence and certainty: distinct probabilistic quantities for different goals. Nat. Neurosci. 19, 366–374 (2016).

34. 34.

Sanders, J. I., Hangya, B. & Kepecs, A. Signatures of a statistical computation in the human sense of confidence. Neuron 90, 499–506 (2016).

35. 35.

van den Berg, R., Yoo, A. H. & Ma, W. J. Fechner’s law in metacognition: a quantitative model of visual working memory confidence. Psychol. Rev. 124, 197–214 (2017).

36. 36.

Wiech, K. Deconstructing the sensation of pain: the influence of cognitive processes on pain perception. Science 354, 584–587 (2016).

37. 37.

Hare, T. A., Camerer, C. F. & Rangel, A. Self-control in decision-making involves modulation of the vmPFC valuation system. Science 324, 646–648 (2009).

38. 38.

Sen, A. K. Choice functions and revealed preference. Rev. Econ. Stud. 38, 307–317 (1971).

39. 39.

Bernheim, B. D. & Rangel, A. Beyond revealed preference: choice-theoretic foundations for behavioral welfare economics. Q. J. Econ. 124, 51–104 (2009).

40. 40.

Sims, C. R. Rate-distortion theory and human perception. Cognition 152, 181–198 (2016).

41. 41.

Stewart, N., Chater, N. & Brown, G. D. A. Decision by sampling. Cognit. Psychol. 53, 1–26 (2006).

42. 42.

Fleming, S. M. & Daw, N. D. Self-evaluation of decision-making: a general Bayesian framework for metacognitive computation. Psychol. Rev. 124, 91–114 (2017).

43. 43.

Weber, E. U. & Johnson, E. J. Mindful judgment and decision making. Annu. Rev. Psychol. 60, 53–85 (2009).

44. 44.

Baxter, M. G. & Murray, E. A. The amygdala and reward. Nat. Rev. Neurosci. 3, 563–573 (2002).

45. 45.

Bowers, J. S. & Davis, C. J. Bayesian just-so stories in psychology and neuroscience. Psychol. Bull. 138, 389–414 (2012).

46. 46.

Griffiths, T. L., Lieder, F. & Goodman, N. D. Rational use of cognitive resources: levels of analysis between the computational and the algorithmic. Top. Cogn. Sci. 7, 217–229 (2015).

47. 47.

Sims, C. R. Efficient coding explains the universal law of generalization in human perception. Science 360, 652–656 (2018).

48. 48.

Li, V., Michael, E., Balaguer, J., Herce Castañón, S. & Summerfield, C. Gain control explains the effect of distraction in human perceptual, cognitive, and economic decision making. Proc. Natl Acad. Sci. USA 115, E8825–E8834 (2018).

49. 49.

Landry, P. & Webb, R. Pairwise normalization: a neuroeconomic theory of multi-attribute choice. SSRN https://doi.org/10.2139/ssrn.2963863 (2017).

50. 50.

Robson, A. J. The biological basis of economic behavior. J. Econ. Lit. 39, 11–33 (2001).

51. 51.

Polanía, R., Moisa, M., Opitz, A., Grueschow, M. & Ruff, C. C. The precision of value-based choices depends causally on fronto-parietal phase coupling. Nat. Commun. 6, 8090 (2015).

52. 52.

Plummer, M. et. al. JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling. In Proc. 3rd International Workshop on Distributed Statistical Computing (DSC2003) (2003).

53. 53.

Gelman, A. et al. Bayesian Data Analysis. 3rd edn, (CRC Press, Boca Raton, 2013).

54. 54.

Venables, W. N. & Ripley, B. D. Modern Applied Statistics with S. (Springer, New York, 2002).

55. 55.

R Core Team R: A Language and Environment for Statistical Computing (R Foundation for Statistical Computing, Vienna, 2017).

56. 56.

Vehtari, A., Gelman, A. & Gabry, J. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Stat. Comput. 27, 1413–1432 (2017).

## Acknowledgements

R.P. thanks X.-X. Wei and A. Stocker for inspiring discussions. We thank S. Maier for providing us with the set of food images and C. Schnyder for research assistance. This work was supported by a grant of the Swiss National Science Foundation (grant IZK0Z1_173607) and an ERC starting grant (ENTRAINER) to R.P; by a grant of the US National Science Foundation to M.W.; and by grants of the Swiss National Science Foundation (grants 105314_152891 and 100019L_173248) and an ERC consolidator grant (BRAINCODES) to C.C.R. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 725355 and No. 758604).

## Author information

### Affiliations

1. #### Zurich Center for Neuroeconomics (ZNE), Department of Economics, University of Zurich, Zurich, Switzerland

• Rafael Polanía
•  & Christian C. Ruff
2. #### Decision Neuroscience Lab, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland

• Rafael Polanía
3. #### Department of Economics, Columbia University, New York, NY, USA

• Rafael Polanía
•  & Michael Woodford

### Contributions

R.P. and C.C.R. designed the study. R.P. collected and analyzed the data. All authors interpreted the results and wrote the manuscript.

### Competing interests

The authors declare no competing interests.

### Corresponding authors

Correspondence to Rafael Polanía or Christian C. Ruff.

## Integrated supplementary information

1. ### Supplementary Figure 1 Schematic illustration of the hierarchical model of the value inference process based on subjective ratings.

For an experimental data set consisting of M goods and N value ratings for each good, we can find the set of parameters of the prior, the internal valuation noise σ, external noise $$\sigma _{{\mathrm{ext}}}$$, and the ‘true’ stimulus values $$v_{(1, \cdots ,M)}$$ that maximize the likelihood of the observed set of ratings under the constraint that $$v_{(1, \cdots ,M)}$$ is distributed following p(v). In our experiments, we parameterized the prior using a logistic distribution (see Methods), however, any other parametrization is possible. Note that the parameters of the prior also constrain the likelihood.

2. ### Supplementary Figure 2 Evidence for the validity of the priors assumed in our work.

a) Empirically observed distribution of subjective value estimates $$\hat v$$ in our experiments. b) Population prior distribution obtained from fits of our model. Comparing this plot to plot a validates the assumed parametrization of the prior in our model. In order to test our assumptions more quantitatively, we tested the hypothesis that the underlying ‘true’ stimulus values can be explained via the parametric distribution of the prior that we assumed here for each participant. We tested this via two methods: The Kolmogorov-Smirnov test and the Cramér-von Mises criterion. We found that the p-value of the null hypothesis test for each participant was greater than 0.05 for 92% and 97% of the participants based on the Kolmogorov-Smirnov test and the Cramér-von Mises criterion, respectively. This confirms that the validity of the parametric shape of the prior assumed here. c) The blue dots in the figure show for an example participant the subjective value estimates $$\hat v$$ for the food items in the unbounded scale (y-axis) plotted against the underlying ‘true’ stimulus values v discovered by our model (x-axis). The red line corresponds to the predicted average subjective value estimates of our model, while the dashed line shows the identity mapping. The model predicts the repulsion of the subjective value estimates (systematic deviations from identity). d) The prior distribution fitted to the example participant in panel c (blue line) and the distribution of subjective value estimates discovered by our model (grey histogram) also show a good agreement.

3. ### Supplementary Figure 3 Rating variability predictions.

We compared the quality of the efficient-coding model fits (left panels) with a simple and flexible model assuming constant Gaussian noise over the rating scale without posing any prior distribution constraints on the values $$v_{(1, \cdots ,M)}$$ (right panels). We found that the efficient-coding model explains the distribution of rating data considerably better than the alternative model for both Experiments 1 (n = 38, panel a) and 2 (n = 37, panel b). The LOO difference is > 500 units in favor of the efficient encoding framework for both models, which statistically confirms that the efficient-coding model captures the empirical variability more accurately. Black dots correspond to the empirical data and error bars in this panel represent the s.e.m. across participants. Model predictions are based on 500 simulated experiments (semi-transparent red lines) where we draw n = 2 ratings for each good and plot rating variability as a function of the mean rating (exactly as derived for the empirical data). The data of each participant is shown in main text Fig. 2b,c.

4. ### Supplementary Figure 4 Ratings Experiment 1.

Distribution of observed ratings $$\mathop{\breve v}$$ on the bounded rating scale for each of the 38 participants. (b) For visualization purposes, we plot the distribution of underlying value estimates $$\hat v$$ on the internal unbounded scale (see main text).

5. ### Supplementary Figure 5 Ratings Experiment 2.

Distribution of observed ratings $$\mathop{\breve v}$$ on the bounded rating scale for each of the 37 participants. (b) For visualization purposes, we plot the distribution of underlying value estimates $$\hat v$$ on the internal unbounded scale (see main text).

6. ### Supplementary Figure 6 Timing analyses Experiment 3.

The scale used for the rating responses was presented immediately after the food image had disappeared; participants (n = 24) were instructed to then enter their rating as fast as possible. The left panel shows the rating response time distribution across all subjects and trials for short (salmon, 0.9 s) and long (purple, 2.6 s) stimulus presentation times. The mean response times for high and low stimulus presentation times were 1.53±0.45 and 1.39±0.4 ms respectively. The small difference between these RTs (0.13 s) was statistically significant (β=0.15±0.04, P<0.001). However, the effective sampling time (image presentation time + response time) was 1.53±0.4 s longer for long exposure times (β=1.5±0.08, P<0.001). Moreover, please recall that several aspects of our design make it very unlikely that participants could control the rate of information sampling to match the presentation times. That is, participants (1) did not know that the presentation time would differ between different food images, (2) did not have advance information how long any given food image would be present on the screen, and (3) were unaware that a second round of ratings with inverted presentation times would take place. Our experimental approach and empirical results therefore support the assumption that participants were able to draw more samples (for example from memory) in the long-presentation-time condition.

7. ### Supplementary Figure 7 Evidence that the repulsion zone is located where the density of the prior is highest (adapted from Fig. 3c).

Empirical biases (points, n = 24) are color-coded as a function of the prior density resulting from our model predictions (see Supplementary Fig. 2). The repulsion zone corresponds to the area where the density is highest, with the bias crossing the x-axis at the highest-density point of the prior. This is line with the prediction of previous work (Wei and Stocker, 2015) that in the case of unimodal priors, repulsion should take place in the vicinity of the prior mode (note that we assume a logistic distribution in our study, which is in indeed a unimodal density function). We also show that the prior starts to exert attraction once the wider likelihood in the high noise regime is located away from the peak of the prior. This is a phenomenon expected based on classical Bayesian frameworks. This difference to the work by Wei & Stocker may reflect that their earlier studies did not consider (near-)boundary effects on non-circular scales, as we considered in our simulations. Error bars in this figure represent s.e.m. across participants. The data of each participant is shown in main text Fig. 3c. Wei, X.-X., and Stocker, A.A. (2015). A Bayesian observer model constrained by efficient coding can explain ’anti-Bayesian’ percepts. Nat. Neurosci. 18, 1509–1517.

8. ### Supplementary Figure 8 Rating biases as a function of different sources of noise.

Simulated differences in value rating as a function of noise for: a) the efficient-coding model (that is, noise in the encoding of the internal value representation). b) Early noise that corrupts the input value signal v0 before it enters the encoding stage (see Supplementary Note 1). c) Late noise in the decision stage (post-decoding noise) that affects the decoded variable and that may capture any unspecific forms of downstream noise unrelated to valuation per se (see Supplementary Note 1). d) The classical Bayes model (that is, without efficient coding). e) Lapses alone (left panel), in combination with external pre-encoding noise (middle panel) or external post-decoding noise (right panel). Color gradients of the simulated data (dots) and interpolated data (lines) represent the results for different levels of lapses in the model simulations (dark to light colors represent low to high lapse rates). We employed the following strategy to derive these predictions. We used similar prior parameters to those obtained in experiments 1 and 2 to randomly draw 50,000 stimulus value inputs v0. We applied the corresponding inference process by corrupting the signals with high and low noise (see panel description, above) to obtain the corresponding subjective value estimates $$\hat v$$ that were subsequently mapped to a rating scale value via $$g(\hat v)$$. We then estimated the mean difference of the ratings for the high-noise and low-noise conditions for each bin, replicating the procedures also implemented for the empirical data (Fig. 3c in main text). Error bars correspond to s.e.m. for each bin and the blue line interpolates these data for visualization. The only model that shows a remarkable overlap with the biases observed in the empirical data is the efficient-coding model (panel a).

9. ### Supplementary Figure 9 Factorial modeling approach that exhaustively tests all possible combinations of noise factors that in principle could explain the observed biases in subjective value ratings under time pressure.

The noise factors that we studied are (see panel a): 1. Pre-encoding noise (pre): Sensory transduction noise before value inference is computed (for example, retinal noise). 2. Efficient Coding noise (EC): Noise resulting from value inference via sampling. This part of noise should be directly affected by presentation time of the food images, which determines the number of effective samples (for example, from memory) that can be drawn and therefore the noisiness of value representations. 3. Post-encoding noise (post): Any form of downstream noise that is not related to value inference per se, for example, motor/muscle noise. 4. Lapse rates (lapse): Quantifies the rate of random decisions due to distraction/lapses during the performance of the valuation task. To run this analysis, we assumed that for ratings with long exposure time, the respective source of noise was nearly zero and evaluated how ‘short exposure times’ affect the respective noise levels that potentially explain the observed ratings. The analysis shows that the models that best explain the data are those that incorporate efficient-coding noise (see Supplementary Table 1). We performed the factorial model comparison by generating the likelihoods that the observed subjective value estimations under time pressure are generated by a given generative model (while appropriately penalizing for model complexity). We formally compare the different models via a log factor likelihood ratio approach (LFLR) that quantifies the degree of belief in each factor (Van Horn, 2003; Shen and Ma, 2018). In brief, we find the marginal likelihood that a factor F is present by marginalizing over all models M in the model space $$L\left( {F_{{\mathrm{present}}}} \right) \approx \mathop {\sum}\limits_M {p\left( {{\mathrm{data|}}M} \right)\left( {M|F_{{\mathrm{present}}}} \right),}$$ while assuming that all models are equally probable. One can analogously find the marginal likelihood of the factor’s absence and then compute the LFLR based via $${\mathrm{LFLR}}_{{\mathrm{AIC/BIC}}}(F) \equiv \log \frac{{p(data|F_{{\mathrm{present}}})}}{{p(data|F_{{\mathrm{absent}}})}}.$$ We approximated the marginal log likelihood of a given model by -0.5 the AIC or BIC of that model. We conducted this analysis independently for each participant. Panel b shows the LRLR results for all noise factors averaged across participants (n = 24) using both AIC (left) and BIC (right) to estimate the likelihood of the data given the fitted parameters. The results clearly indicate that the internal noise of the efficient-coding (EC) model is the only factor significantly explaining the data. Horizontal dashed lines represent the levels of evidence for a given LFLR. The average LFLRs of the EC factor are >9.6, which corresponds to a Bayes factor BF>100. This provides overwhelming evidence for the factor being relevant (Jeffreys, 1961). No other factor crosses the moderate evidence line. Error bars in this panel indicate s.e.m. Panel c shows the LFLRs of the EC factor for each participant. This analysis reveals a positive LFLRs for the large majority (21 out of 24) of our participants. These results provide compelling evidence that manipulation of time exposure (when valuating food items) affects internal noise via efficient encoding. Van Horn, K.S. (2003). Constructing a logic of plausible inference: a guide to Cox’s theorem. Int. J. Approx. Reason. 34, 3–24. Jeffreys, H. (1961). Theory of probability (Oxford: Clarendon Press). Shen, S., and Ma, W.J. (2018). Variable precision in visual perception. Preprint at bioRxiv 153650.

## Supplementary information

1. ### Supplementary Figures 1–9

Supplementary Figs. 1–9 and the Supplementary Tables and Notes file.