Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • News & Views
  • Published:

Neural networks

Deconstructing the generalization gap

New research reveals a duality between neural network weights and neuron activities that enables a geometric decomposition of the generalization gap. The framework provides a way to interpret the effects of regularization schemes such as stochastic gradient descent and dropout on generalization — and to improve upon these methods.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Two methods to reduce the generalization gap.


  1. Neyshabur, B., Tomioka, R. & Srebro, N. Preprint at (2014).

  2. Zhang, C., Bengio, S., Hardt, M., Recht, B. & Vinyals, O. In Int. Conf. Learning Representations (ICLR) 2017 (2022).

  3. Feng, Y., Zhang, W. & Tu, Y. Nat. Mach. Intell. 5, 908–918 (2023).

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Andrey Gromov.

Ethics declarations

Competing interests

The author declares no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gromov, A. Deconstructing the generalization gap. Nat Mach Intell 5, 1340–1341 (2023).

Download citation

  • Published:

  • Issue Date:

  • DOI:


Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics