Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Human generalization of internal representations through prototype learning with goal-directed attention

Nature Human Behaviour volume 7, pages 442–463 (2023)Cite this article

Subjects

Abstract

The world is overabundant with feature-rich information obscuring the latent causes of experience. How do people approximate the complexities of the external world with simplified internal representations that generalize to novel examples or situations? Theories suggest that internal representations could be determined by decision boundaries that discriminate between alternatives, or by distance measurements against prototypes and individual exemplars. Each provide advantages and drawbacks for generalization. We therefore developed theoretical models that leverage both discriminative and distance components to form internal representations via action-reward feedback. We then developed three latent-state learning tasks to test how humans use goal-oriented discrimination attention and prototypes/exemplar representations. The majority of participants attended to both goal-relevant discriminative features and the covariance of features within a prototype. A minority of participants relied only on the discriminative feature. Behaviour of all participants could be captured by parameterizing a model combining prototype representations with goal-oriented discriminative attention.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Distance and discriminative models of latent-state learning produce distinct generalization failures.
Fig. 2: Algorithmic models of latent-state instrumental reinforcement learning using prototype or exemplar with discriminative attention.
Fig. 3: Experiment 1. Human participants use discriminative attention when generalizing to novel examples.
Fig. 4: Experiment 1. Varying algorithmic model use of top-down discriminative attention recapitulates the spectrum of human behaviour.
Fig. 5: Experiment 2. Majority of participants use previously non-discriminative features to generalize in novel context. A subpopulation attends to one discriminative feature.
Fig. 6: Experiment 3. Majority of participants attend to common state features.
Fig. 7: Reproducing the discriminative feature attention subpopulation through alteration of algorithmic model context recognition.
Fig. 8: The majority of human participants and the ProDAtt model attend to prototype covariance.

Data availability

Behavioural data are available at https://zenodo.org/record/7186975. Source data are provided with this paper.

Code availability

The simulation code and behavioural analysis code are available at https://github.com/murraylab/instrumentalLatentStateLearning. The Jupyter notebook designed to aid in model intuition is available through the simulation repository, and is hosted online through a link from its own repository: https://github.com/murraylab/ProDAttExDAttModelIntuition. Task code is available at https://github.com/murraylab/alienArtifactsLearningTask.

References

  1. Gershman, S. J., Blei, D. M. & Niv, Y. Context, learning, and extinction. Psychol. Rev. 117, 197–209 (2010).

    Article  PubMed  Google Scholar 

  2. Redish, A. D., Jensen, S., Johnson, A. & Kurth-Nelson, Z. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol. Rev. 114, 784–805 (2007).

    Article  PubMed  Google Scholar 

  3. Gershman, S. J. & Niv, Y. Exploring a latent cause theory of classical conditioning. Learn. Behav. 40, 255–268 (2012).

    Article  PubMed  Google Scholar 

  4. Collins, A. & Koechlin, E. Reasoning, learning, and creativity: frontal lobe function and human decision-making. PLOS Biol. 10, e1001293 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Gershman, S. J., Monfils, M.-H., Norman, K. A. & Niv, Y. The computational nature of memory modification. eLife 6, e23763 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Cochran, A. L. & Cisler, J. M. A flexible and generalizable model of online latent-state learning. PLOS Comput. Biol. 15, e1007331 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Ashby, F. G. & Maddox, W. T. Relations between prototype, exemplar, and decision bound models of categorization. J. Math. Psychol. 37, 372–400 (1993).

    Article  Google Scholar 

  8. Ashby, F. G. & Gott, R. E. Decision rules in the perception and categorization of multidimensional stimuli. J. Exp. Psychol. Learn. Mem. Cogn. 14, 33–53 (1988).

    Article  CAS  PubMed  Google Scholar 

  9. Reed, S. K. Pattern recognition and categorization. Cogn. Psychol. 3, 382–407 (1972).

    Article  Google Scholar 

  10. Reed, S. K. & Friedman, M. P. Perceptual vs conceptual categorization. Mem. Cogn. 1, 157–163 (1973).

    Article  CAS  Google Scholar 

  11. Nosofsky, R. M. Tests of an exemplar model for relating perceptual classification and recognition memory. J. Exp. Psychol. Hum. Percept. Perform. 17, 3–27 (1991).

    Article  CAS  PubMed  Google Scholar 

  12. Smith, J. D., Murray, M. J. & Minda, J. P. Straight talk about linear separability. J. Exp. Psychol. Learn. Mem. Cogn 23, 659–680 (1997).

    Article  Google Scholar 

  13. Smith, J. D. Prototypes in the mist: the early epochs of category learning. J. Exp. Psychol. Learn. Mem. Cogn 24, 1411–1436 (1998).

    Article  Google Scholar 

  14. Kruschke, J. K. ALCOVE: an exemplar-based connectionist model of category learning. Psychol. Rev. 99, 22–44 (1992).

    Article  CAS  PubMed  Google Scholar 

  15. Kruschke, J. K. & Erickson, M. A. Learning of rules that have high-frequency exceptions: new empirical data and a hybrid connectionist model. In Proc. Sixteenth Annual Conference of the Cognitive Science Society (eds Ram, A. & Eiselt, K.) 514–519 (Routledge, 1994).

  16. Kruschke, J. K. Base rates in category learning. J. Exp. Psychol. Learn. Mem. Cogn. 22, 3–26 (1996).

    Article  CAS  PubMed  Google Scholar 

  17. Kruschke, J. K. Toward a unified model of attention in associative learning. J. Math. Psychol. 45, 812–863 (2001).

    Article  Google Scholar 

  18. Kruschke, J. K. & Johansen, M. K. A model of probabilistic category learning. J. Exp. Psychol. Learn. Mem. Cogn. 25, 1083–1119 (1999).

    Article  CAS  PubMed  Google Scholar 

  19. Nosofsky, R. M., Palmeri, T. J. & McKinley, S. C. Rule-plus-exception model of classification learning. Psychol. Rev. 101, 53–79 (1994).

    Article  CAS  PubMed  Google Scholar 

  20. Ashby, F. G. & Maddox, W. T. Human category learning. Annu. Rev. Psychol. 56, 149–178 (2005).

    Article  PubMed  Google Scholar 

  21. Minda, J. P. & David, S. J. Prototypes in category learning: the effects of category size, category structure, and stimulus complexity. J. Exp. Psychol. Learn. Mem. Cogn 27, 775–799 (2001).

    Article  CAS  PubMed  Google Scholar 

  22. Smith, D. J. & Minda, J. P. Thirty categorization results in search of a model. J. Exp. Psychol. Learn. Mem. Cogn. 26, 3–27 (2000).

    Article  CAS  PubMed  Google Scholar 

  23. Medin, D. L. & Schaffer, M. M. Context theory of classification learning. Psychol. Rev. 85, 207–238 (1978).

    Article  Google Scholar 

  24. Estes, W. K. Array models for category learning. Cogn. Psychol. 18, 500–549 (1986).

    Article  CAS  PubMed  Google Scholar 

  25. Nosofsky, R. M. Choice, similarity, and the context theory of classification. J. Exp. Psychol. Learn. Mem. Cogn. 10, 104–114 (1984).

    Article  CAS  PubMed  Google Scholar 

  26. Nosofsky, R. M. Attention, similarity, and the identification-categorization relationship. J. Exp. Psychol. Gen 115, 39–57 (1986).

    Article  CAS  PubMed  Google Scholar 

  27. Ashby, F. G. in Multidimensional Models of Perception and Cognition 449–483 (Scientific Psychology Series, Lawrence Erlbaum Associates, 1992).

  28. Maddox, W. T. & Ashby, F. G. Comparing decision bound and exemplar models of categorization. Percept. Psychophys. 53, 49–70 (1993).

    Article  CAS  PubMed  Google Scholar 

  29. Niv, Y. et al. Reinforcement learning in multidimensional environments relies on attention mechanisms. J. Neurosci. 35, 8145–8157 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Love, B. C., Medin, D. L. & Gureckis, T. M. SUSTAIN: a network model of category learning. Psychol. Rev. 111, 309–332 (2004).

    Article  PubMed  Google Scholar 

  31. French, R. M. Catastrophic forgetting in connectionist networks. Trends Cogn. Sci. 3, 128–135 (1999).

    Article  CAS  PubMed  Google Scholar 

  32. Flesch, T., Balaguer, J., Dekker, R., Nili, H. & Summerfield, C. Comparing continual task learning in minds and machines. Proc. Natl. Acad. Sci. USA 115, E10313–E10322 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Parisi, G. I., Kemker, R., Part, J. L., Kanan, C. & Wermter, S. Continual lifelong learning with neural networks: a review. Neural Netw. 113, 54–71 (2019).

    Article  PubMed  Google Scholar 

  34. Corbetta, M. & Shulman, G. L. Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci. 3, 201–215 (2002).

    Article  CAS  PubMed  Google Scholar 

  35. Noudoost, B., Chang, M. H., Steinmetz, N. A. & Moore, T. Top-down control of visual attention. Curr. Opin. Neurobiol. 20, 183–190 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Baluch, F. & Itti, L. Mechanisms of top-down attention. Trends Neurosci. 34, 210–224 (2011).

    Article  CAS  PubMed  Google Scholar 

  37. Bhatnagar, S., Sutton, R., Ghavamzadeh, M. & Lee, M. Natural Actor-Critic Algorithms. Automatica 45, 2471–2482 (2009).

    Article  Google Scholar 

  38. Casler, K., Bickel, L. & Hackett, E. Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Comput. Hum. Behav. 29, 2156–2160 (2013).

    Article  Google Scholar 

  39. Peer, E., Brandimarte, L., Samat, S. & Acquisti, A. Beyond the Turk: alternative platforms for crowdsourcing behavioral research. J. Exp. Soc. Psychol. 70, 153–163 (2017).

    Article  Google Scholar 

  40. Vehtari, A., Gelman, A. & Gabry, J. Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Stat. Comput. 27, 1413–1432 (2017).

    Article  Google Scholar 

  41. Kumar, R., Carroll, C., Hartikainen, A. & Martin, O. ArviZ a unified library for exploratory analysis of Bayesian models in Python. J. Open Source Softw. 4, 1143 (2019).

    Article  Google Scholar 

  42. Barak, O., Rigotti, M. & Fusi, S. The sparseness of mixed selectivity neurons controls the generalization-discrimination trade-off. J. Neurosci. 33, 3844–3856 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Bernardi, S. et al. The geometry of abstraction in the hippocampus and prefrontal cortex. Cell 183, 954–967.e21 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  44. Ito, T. & Murray, J. D. Multi-task representations in human cortex transform along a sensory-to-motor hierarchy (2021).

  45. Collins, A. G. E. & Frank, M. J. How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis. Eur. J. Neurosci. 35, 1024–1035 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  46. Collins, A. G. E., Brown, J. K., Gold, J. M., Waltz, J. A. & Frank, M. J. Working memory contributions to reinforcement learning impairments in schizophrenia. J. Neurosci. 34, 13747–13756 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Collins, A. G. E., Ciullo, B., Frank, M. J. & Badre, D. Working memory load strengthens reward prediction errors. J. Neurosci. 37, 4332–4342 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Mack, M. L., Love, B. C. & Preston, A. R. Dynamic updating of hippocampal object representations reflects new conceptual knowledge. Proc. Natl. Acad. Sci. USA 113, 13203–13208 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Mack, M. L., Preston, A. R. & Love, B. C. Ventromedial prefrontal cortex compression during concept learning. Nat. Commun. 11, 46 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Flesch, T., Nagy, D. G., Saxe, A. & Summerfield, C. Modelling continual learning in humans with Hebbian context gating and exponentially decaying task signals. PLOS Comput. Bio. 19, e1010808 (2023).

    Article  CAS  Google Scholar 

  51. Heald, J. B., Lengyel, M. & Wolpert, D. M. Contextual inference underlies the learning of sensorimotor repertoires. Nature 600, 489–493 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Miller, H. L., Ragozzino, M. E., Cook, E. H., Sweeney, J. A. & Mosconi, M. W. Cognitive set shifting deficits and their relationship to repetitive behaviors in autism spectrum disorder. J. Autism Dev. Disord. 45, 805–815 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  53. Lawson, R. P., Mathys, C. & Rees, G. Adults with autism overestimate the volatility of the sensory environment. Nat. Neurosci. 20, 1293–1299 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  54. Friedman-Hill, S. R. et al. What does distractibility in ADHD reveal about mechanisms for top-down attentional control? Cognition 115, 93–103 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  55. Ramos, A. A., Hamdan, A. C. & Machado, L. A meta-analysis on verbal working memory in children and adolescents with ADHD. Clin. Neuropsychol. 34, 873–898 (2020).

    Article  PubMed  Google Scholar 

  56. Taurines, R. et al. ADHD and autism: differential diagnosis or overlapping traits? A selective review. Atten. Defic. Hyperact. Disord. 4, 115–139 (2012).

    Article  PubMed  Google Scholar 

  57. Kern, J. K., Geier, D. A., Sykes, L. K., Geier, M. R. & Deth, R. C. Are ASD and ADHD a continuum? A comparison of pathophysiological similarities between the disorders. J. Atten. Disord. 19, 805–827 (2015).

    Article  PubMed  Google Scholar 

  58. Antshel, K. M., Zhang-James, Y., Wagner, K. E., Ledesma, A. & Faraone, S. V. An update on the comorbidity of ADHD and ASD: a focus on clinical management. Expert Rev. Neurother. 16, 279–293 (2016).

    Article  CAS  PubMed  Google Scholar 

  59. Friston, K. The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11, 127–138 (2010).

    Article  CAS  PubMed  Google Scholar 

  60. Myung, I. J. Maximum entropy interpretation of decision bound and context models of categorization. J. Math. Psychol. 38, 335–365 (1994).

    Article  Google Scholar 

  61. Abadi, M. et al. TensorFlow: large-scale machine learning on heterogeneous distributed systems. Preprint at arXiv https://doi.org/10.48550/arXiv.1603.04467 (2016).

  62. Flesch, T., Juechems, K., Dumbalska, T., Saxe, A. & Summerfield, C. Orthogonal representations for robust context-dependent task performance in brains and neural networks. Neuron 110, 1258–1270 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  63. Capretto, T., Piho, C., Kumar, R., Westfall, J., Yarkoni, T., & Martin, O. A. Bambi: a simple interface for fitting Bayesian linear models in Python. J. Stat. Softw. 103, 1–29 (2022).

  64. Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261–272 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  65. Morey, R. D., Rouder, J. N., Jamil, T. & Morey, M. R. D. Package ‘bayesfactor.’ http://www.cran/r-projectorg/web/packages/BayesFactor/BayesFactorpdfi (2015).

Download references

Acknowledgements

This research was partly supported by NIH grants R01MH112746 (J.D.M.), R01MH112688 and P50MH119569 (A.D.R.), a SFARI Human Cognitive and Behavioural Science Explorer Award 988485 (J.D.M.), as well as by an appointment to the Intelligence Community Postdoctoral Research Fellowship Program at Yale University, administered by Oak Ridge Institute for Science and Education (ORISE) through an interagency agreement between the US Department of Energy and the Office of the Director of National Intelligence (ODNI) (W.W.P.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We thank V. Gouder, A. Di Martino, D. Ehrlich and M. Jones for comments and discussion, and the Marine Biological Laboratory Methods in Computational Neuroscience course, where this research was initiated.

Author information

Author notes

  1. These authors jointly supervised this work: A. David Redish and John D. Murray.

Authors and Affiliations

  1. Department of Psychiatry, Yale School of Medicine, New Haven, CT, USA

    Warren Woodrich Pettine & John D. Murray

  2. Department of Informatics, University of Sussex, Brighton, UK

    Dhruva Venkita Raman

  3. Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA

    A. David Redish

Authors
  1. Warren Woodrich Pettine
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dhruva Venkita Raman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. David Redish
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. John D. Murray
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

W.W.P., D.V.R., A.D.R. and J.D.M. conceptualized the project. W.W.P., D.V.R. and A.D.R. developed the theoretical model. W.W.P. conducted behavioural investigation and formal analysis, developed the code base and wrote the original draft. W.W.P., A.D.R. and J.D.M. reviewed and edited the manuscript. W.W.P., A.D.R. and J.D.M. acquired funding. A.D.R. and J.D.M supervised the project.

Corresponding author

Correspondence to John D. Murray.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Fuat Balcı and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–8, Tables 1 and 2 and Notes A–C.

Reporting Summary

Supplementary Table

Full statistical reports.

Source data

Source Data Fig. 3

Metrics for each participant, used to create histograms in Fig. 3f.

Source Data Fig. 5

Metrics for the histograms and the error count for the confusion matrices, including tabs for data pertaining to Fig. 5d,f.

Source Data Fig. 6

Metrics for the histograms and the error count for the confusion matrices, including tabs for data pertaining to Fig. 6b,d.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pettine, W.W., Raman, D.V., Redish, A.D. et al. Human generalization of internal representations through prototype learning with goal-directed attention. Nat Hum Behav 7, 442–463 (2023). https://doi.org/10.1038/s41562-023-01543-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-023-01543-7

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing