Bioinspired learning

Moving beyond reward prediction errors

Classic theories of reinforcement learning and neuromodulation rely on reward prediction errors. A new machine learning technique relies on neuromodulatory signals that are optimized for specific tasks, which may lead to better AI and better explanations of neuroscience data.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Standard models of dopamine versus backpropamine.

References

  1. 1.

    Hassabis, D., Kumaran, D., Summerfield, C. & Botvinick, M. Neuron 95, 245–258 (2017).

    Article  Google Scholar 

  2. 2.

    Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 1998).

  3. 3.

    Miconi, T., Rawal, A., Clune, J. & Stanley, K. O. in Int. Conf. Learning Representations (ICLR, 2019).

  4. 4.

    Sutton, R. S. & Barto, A. G. Psychol. Rev. 88, 135–170 (1981).

    Article  Google Scholar 

  5. 5.

    Rescorla, R. A. & Wagner, A. R. in Classical Conditioning II: Current Research and Theory (eds Black, A. H., & Prokasy, W. F.) 64–99 (Appleton-Century-Crofts, 1972).

  6. 6.

    Silver, D. et al. Nature 529, 484–489 (2016).

    Article  Google Scholar 

  7. 7.

    Mnih, V. et al. Nature 518, 529–533 (2015).

    Article  Google Scholar 

  8. 8.

    Iversen, S. D. & Iversen, L. L. Trends Neurosci. 30, 188–193 (2007).

    Article  Google Scholar 

  9. 9.

    Schultz, W., Dayan, P. & Montague, P. R. Science 275, 1593–1599 (1997).

    Article  Google Scholar 

  10. 10.

    Brzosko, Z., Schultz, W. & Paulsen, O. eLife 4, e09685 (2015).

    Article  Google Scholar 

  11. 11.

    Roelfsema, P. R. & Holtmaat, A. Nat. Rev. Neurosci. 19, 166–180 (2018).

    Article  Google Scholar 

  12. 12.

    Frémaux, N. & Gerstner, W. Front. Neural Circuits 9, 85 (2016).

    Article  Google Scholar 

  13. 13.

    Sharpe, M. J. et al. Nat. Neurosci. 20, 735–742 (2017).

    Article  Google Scholar 

  14. 14.

    Takahashi, Y. K. et al. Neuron 95, 1395–1405 (2017).

    Article  Google Scholar 

  15. 15.

    Coddington, L. T. & Dudman, J. T. Nat. Neurosci. 21, 1563–1573 (2018).

    Article  Google Scholar 

  16. 16.

    Andrychowicz, M. et al. in 30th Conf. Neural Information Processing Systems 3981–3989 (NIPS, 2016).

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Blake A. Richards.

Ethics declarations

Competing interests

The author declares no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Richards, B.A. Moving beyond reward prediction errors. Nat Mach Intell 1, 204–205 (2019). https://doi.org/10.1038/s42256-019-0053-0

Download citation

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing