Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Neural computation

A ‘programming’ framework for recurrent neural networks

Nature Machine Intelligence volume 5, pages 570–571 (2023)Cite this article

Subjects

A ‘programming’-like approach provides a one-step algorithm to find network parameters for recurrent neural networks that can model complex dynamical systems.

Recurrent neural networks (RNNs) are foundational tools for performing tasks that require time-dependent computation. They make use of recurrent connections between individual network units to process sequential inputs and generate a wide range of dynamics. Moreover, they are ideal for tasks that require extensive distributed parallel processing such as determining whether a given maze has a solution1. However, the major obstacle to using RNNs for such complex computations is finding the right set of network parameters for each task. The most common approaches optimize recurrent networks via supervised training algorithms in which the network is simulated many times, processing a large number of different input–output pairs, and slowly correcting errors. In this issue of Nature Machine Intelligence, Kim and Bassett focus on an alternative approach, adopting tools from neuroscience and software development2. Their work presents an innovative method to design RNNs that are capable of performing complex computations based on the direct engineering of network parameters. Unlike standard training techniques, this one-step algorithm determines appropriate network parameters analytically, effectively ‘programming’ network computations in a manner analogous to how programming languages work on computer hardware.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Programming of a recurrent neural network.

References

  1. Larsen, B. W. & Druckmann, S. PLoS Comput. Biol. 18, e1010227 (2022).

    Article  Google Scholar 

  2. Kim, J. Z. & Bassett, D. S. Nat. Mach. Intell. https://doi.org/10.1038/s42256-023-00668-8 (2023).

  3. Doya, K. IEEE Transactions on Neural Networks 1, 1–6 (1993).

    Google Scholar 

  4. Maass, W., Joshi, P. & Sontag, E. D. PLoS Comput. Biol. 3, e165 (2007).

    Article  Google Scholar 

  5. Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In Proc. 3rd International Conference on Learning Representations (ICLR) (ICLR, 2015)

  6. Paszke, A. et al. NIPS 2017 Autodiff Workshop (2017).

  7. Eliasmith, C. & Anderson, C. H. Neural engineering: Computation, representation, and dynamics in neurobiological systems. (MIT Press, 2003).

  8. Beiran, M., Dubreuil, A., Valente, A., Mastrogiuseppe, F. & Ostojic, S. Neural Comput. 33, 1572–1615 (2021).

    Article  MathSciNet  Google Scholar 

  9. Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T. & Wang, X.-J. Nat. Neurosci. 22, 297–306 (2019).

    Article  Google Scholar 

  10. Márton, C. D., Gagnon, L., Lajoie, G. & Rajan, K. Preprint at arXiv https://doi.org/10.48550/arXiv.2105.14108 (2021).

Download references

Author information

Authors and Affiliations

  1. Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA

    Manuel Beiran

  2. Nash Family Department of Neuroscience, Icahn School of Medicine, Mount Sinai, New York, NY, USA

    Manuel Beiran, Camille A. Spencer-Salmon & Kanaka Rajan

Authors
  1. Manuel Beiran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Camille A. Spencer-Salmon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kanaka Rajan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Manuel Beiran or Kanaka Rajan.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Beiran, M., Spencer-Salmon, C.A. & Rajan, K. A ‘programming’ framework for recurrent neural networks. Nat Mach Intell 5, 570–571 (2023). https://doi.org/10.1038/s42256-023-00674-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-023-00674-w

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing