Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • News & Views
  • Published:

Bioinspired learning

Moving beyond reward prediction errors

Classic theories of reinforcement learning and neuromodulation rely on reward prediction errors. A new machine learning technique relies on neuromodulatory signals that are optimized for specific tasks, which may lead to better AI and better explanations of neuroscience data.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Standard models of dopamine versus backpropamine.


  1. Hassabis, D., Kumaran, D., Summerfield, C. & Botvinick, M. Neuron 95, 245–258 (2017).

    Article  Google Scholar 

  2. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 1998).

  3. Miconi, T., Rawal, A., Clune, J. & Stanley, K. O. in Int. Conf. Learning Representations (ICLR, 2019).

  4. Sutton, R. S. & Barto, A. G. Psychol. Rev. 88, 135–170 (1981).

    Article  Google Scholar 

  5. Rescorla, R. A. & Wagner, A. R. in Classical Conditioning II: Current Research and Theory (eds Black, A. H., & Prokasy, W. F.) 64–99 (Appleton-Century-Crofts, 1972).

  6. Silver, D. et al. Nature 529, 484–489 (2016).

    Article  Google Scholar 

  7. Mnih, V. et al. Nature 518, 529–533 (2015).

    Article  Google Scholar 

  8. Iversen, S. D. & Iversen, L. L. Trends Neurosci. 30, 188–193 (2007).

    Article  Google Scholar 

  9. Schultz, W., Dayan, P. & Montague, P. R. Science 275, 1593–1599 (1997).

    Article  Google Scholar 

  10. Brzosko, Z., Schultz, W. & Paulsen, O. eLife 4, e09685 (2015).

    Article  Google Scholar 

  11. Roelfsema, P. R. & Holtmaat, A. Nat. Rev. Neurosci. 19, 166–180 (2018).

    Article  Google Scholar 

  12. Frémaux, N. & Gerstner, W. Front. Neural Circuits 9, 85 (2016).

    Article  Google Scholar 

  13. Sharpe, M. J. et al. Nat. Neurosci. 20, 735–742 (2017).

    Article  Google Scholar 

  14. Takahashi, Y. K. et al. Neuron 95, 1395–1405 (2017).

    Article  Google Scholar 

  15. Coddington, L. T. & Dudman, J. T. Nat. Neurosci. 21, 1563–1573 (2018).

    Article  Google Scholar 

  16. Andrychowicz, M. et al. in 30th Conf. Neural Information Processing Systems 3981–3989 (NIPS, 2016).

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Blake A. Richards.

Ethics declarations

Competing interests

The author declares no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Richards, B.A. Moving beyond reward prediction errors. Nat Mach Intell 1, 204–205 (2019).

Download citation

  • Published:

  • Issue Date:

  • DOI:


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing