Abstract
Motor cortex (M1) exhibits a rich repertoire of neuronal activities to support the generation of complex movements. Although recent neuronalnetwork models capture many qualitative aspects of M1 dynamics, they can generate only a few distinct movements. Additionally, it is unclear how M1 efficiently controls movements over a wide range of shapes and speeds. We demonstrate that modulation of neuronal input–output gains in recurrent neuronalnetwork models with a fixed architecture can dramatically reorganize neuronal activity and thus downstream muscle outputs. Consistent with the observation of diffuse neuromodulatory projections to M1, a relatively small number of modulatory control units provide sufficient flexibility to adjust highdimensional network activity using a simple rewardbased learning rule. Furthermore, it is possible to assemble novel movements from previously learned primitives, and one can separately change movement speed while preserving movement shape. Our results provide a new perspective on the role of modulatory systems in controlling recurrent cortical activity.
Access options
Subscribe to Journal
Get full journal access for 1 year
$225.00
only $18.75 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
from$8.99
All prices are NET prices.
Change history
19 December 2018
In the version of this article initially published, in the PDF, equations (2) and (4) erroneously displayed a curly bracket on the right hand side of the equation. This should not be there. The errors have been corrected in the PDF version of the article. The equations appear correctly in the HTML.
References
 1.
Rathelot, J.A. & Strick, P. L. Subdivisions of primary motor cortex based on corticomotoneuronal cells. Proc. Natl. Acad. Sci. USA 106, 918–923 (2009).
 2.
Rosenbaum, D. A. Human Motor Control. (Academic Press, Cambridge, MA,USA, 2009).
 3.
Sanes, J. N. & Donoghue, J. P. Plasticity and primary motor cortex. Annu. Rev. Neurosci. 23, 393–415 (2000).
 4.
Churchland, M. M. et al. Neural population dynamics during reaching. Nature 487, 51–56 (2012).
 5.
Shenoy, K. V., Sahani, M. & Churchland, M. M. Cortical control of arm movements: a dynamical systems perspective. Annu. Rev. Neurosci. 36, 337–359 (2013).
 6.
Afshar, A. et al. Singletrial neural correlates of arm movement preparation. Neuron 71, 555–564 (2011).
 7.
Churchland, M. M., Cunningham, J. P., Kaufman, M. T., Ryu, S. I. & Shenoy, K. V. Cortical preparatory activity: representation of movement or first cog in a dynamical machine? Neuron 68, 387–400 (2010).
 8.
Russo, A. A. et al. Motor cortex embeds musclelike commands in an untangled population response. Neuron 97, 953–966.e8 (2018).
 9.
Churchland, M. M. & Cunningham, J. P. A dynamical basis set for generating reaches. Cold Spring Harb. Symp. Quant. Biol. 79, 67–80 (2014).
 10.
Sussillo, D., Churchland, M. M., Kaufman, M. T. & Shenoy, K. V. A neural network that finds a naturalistic solution for the production of muscle activity. Nat. Neurosci. 18, 1025–1033 (2015).
 11.
Hennequin, G., Vogels, T. P. & Gerstner, W. Optimal control of transient dynamics in balanced networks supports generation of complex movements. Neuron 82, 1394–1406 (2014).
 12.
Sehgal, M., Song, C., Ehlers, V. L. & Moyer, J. R. Jr. Learning to learn  intrinsic plasticity as a metaplasticity mechanism for memory formation. Neurobiol. Learn. Mem. 105, 186–199 (2013).
 13.
Kida, H. & Mitsushima, D. Mechanisms of motor learning mediated by synaptic plasticity in rat primary motor cortex. Neurosci. Res. 128, 14–18 (2018).
 14.
Chance, F. S., Abbott, L. F. & Reyes, A. D. Gain modulation from background synaptic input. Neuron 35, 773–782 (2002).
 15.
Swinehart, C. D., Bouchard, K., Partensky, P. & Abbott, L. F. Control of network activity through neuronal response modulation. Neurocomputing 58–60, 327–335 (2004).
 16.
Zhang, J. & Abbott, L. F. Gain modulation of recurrent networks. Neurocomputing 32–33, 623–628 (2000).
 17.
Marder, E. Neuromodulation of neuronal circuits: back to the future. Neuron 76, 1–11 (2012).
 18.
Salinas, E. & Thier, P. Gain modulation: a major computational principle of the central nervous system. Neuron 27, 15–21 (2000).
 19.
MolinaLuna, K. et al. Dopamine in motor cortex is necessary for skill learning and synaptic plasticity. PLoS One 4, e7082 (2009).
 20.
Thurley, K., Senn, W. & Lüscher, H.R. Dopamine increases the gain of the inputoutput response of rat prefrontal pyramidal neurons. J. Neurophysiol. 99, 2985–2997 (2008).
 21.
Vestergaard, M. & Berg, R. W. Divisive gain modulation of motoneurons by inhibition optimizes muscular control. J. Neurosci. 35, 3711–3723 (2015).
 22.
Wei, K. et al. Serotonin affects movement gain control in the spinal cord. J. Neurosci. 34, 12690–12700 (2014).
 23.
Hosp, J. A., Pekanovic, A., RioultPedotti, M. S. & Luft, A. R. Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J. Neurosci. 31, 2481–2487 (2011).
 24.
Huntley, G. W., Morrison, J. H., Prikhozhan, A. & Sealfon, S. C. Localization of multiple dopamine receptor subtype mRNAs in human and monkey motor cortex and striatum. Brain Res. Mol. Brain Res. 15, 181–188 (1992).
 25.
Thoroughman, K. A. & Shadmehr, R. Learning of action through adaptive combination of motor primitives. Nature 407, 742–747 (2000).
 26.
Giszter, S. F. Motor primitivesnew data and future questions. Curr. Opin. Neurobiol. 33, 156–165 (2015).
 27.
Lara, A. H., Cunningham, J. P. & Churchland, M. M. Different population dynamics in the supplementary motor area and motor cortex during reaching. Nat. Commun. 9, 2754 (2018).
 28.
Rajan, K., Abbott, L. F. & Sompolinsky, H. Stimulusdependent suppression of chaos in recurrent neural networks. Phys. Rev. E 82, 011903 (2010).
 29.
Mazzoni, P., Andersen, R. A. & Jordan, M. I. A more biologically plausible learning rule for neural networks. Proc. Natl. Acad. Sci. USA 88, 4433–4437 (1991).
 30.
Legenstein, R., Chase, S. M., Schwartz, A. B. & Maass, W. A rewardmodulated hebbian learning rule can explain experimentally observed network reorganization in a brain control task. J. Neurosci. 30, 8400–8410 (2010).
 31.
Hoerzer, G. M., Legenstein, R. & Maass, W. Emergence of complex computational structures from chaotic neural networks through rewardmodulated Hebbian learning. Cereb. Cortex 24, 677–690 (2014).
 32.
Miconi, T. Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. eLife 6, e20899 (2017).
 33.
Li, N., Chen, T.W., Guo, Z. V., Gerfen, C. R. & Svoboda, K. A motor cortex circuit for motor planning and movement. Nature 519, 51–56 (2015).
 34.
Sussillo, D. & Abbott, L. F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009).
 35.
Spampinato, D. A., Block, H. J. & Celnik, P. A. Cerebellar–M1 connectivity changes associated with motor learning are somatotopic specific. J. Neurosci. 37, 2377–2386 (2017).
 36.
Kao, J. C. et al. Singletrial dynamics of motor cortex and their applications to brainmachine interfaces. Nat. Commun. 6, 7759 (2015).
 37.
Wang, J., Narain, D., Hosseini, E. A. & Jazayeri, M. Flexible timing by temporal scaling of cortical responses. Nat. Neurosci. 21, 102–110 (2018).
 38.
Soares, S., Atallah, B. V. & Paton, J. J. Midbrain dopamine neurons control judgment of time. Science 354, 1273–1277 (2016).
 39.
Hardy, N. F., Goudar, V., RomeroSosa, J. L. & Buonomano, D. V. A model of temporal scaling correctly predicts that motor timing improves with speed. Nat. Commun. 9, 4732 (2018).
 40.
Collier, G. L. & Wright, C. E. Temporal rescaling of simple and complex ratios in rhythmic tapping. J. Exp. Psychol. Hum. Percept. Perform. 21, 602–627 (1995).
 41.
Gallego, J. A., Perich, M. G., Miller, L. E. & Solla, S. A. Neural manifolds for the control of movement. Neuron 94, 978–984 (2017).
 42.
Friston, K. J. Functional and effective connectivity: a review. Brain Connect. 1, 13–36 (2011).
 43.
Sussillo, D. & Barak, O. Opening the black box: lowdimensional dynamics in highdimensional recurrent neural networks. Neural Comput. 25, 626–649 (2013).
 44.
Kambara, H., Shin, D. & Koike, Y. A computational model for optimal muscle activity considering muscle viscoelasticity in wrist movements. J. Neurophysiol. 109, 2145–2160 (2013).
 45.
Martins, A. R. O. & Froemke, R. C. Coordinated forms of noradrenergic plasticity in the locus coeruleus and primary auditory cortex. Nat. Neurosci. 18, 1483–1492 (2015).
 46.
Swinehart, C. D. & Abbott, L. F. Supervised learning through neuronal response modulation. Neural Comput. 17, 609–631 (2005).
 47.
Breakspear, M. Dynamic models of largescale brain activity. Nat. Neurosci. 20, 340–352 (2017).
 48.
Mante, V., Sussillo, D., Shenoy, K. V. & Newsome, W. T. Contextdependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013).
 49.
Bargmann, C. I. Beyond the connectome: how neuromodulators shape neural circuits. BioEssays 34, 458–465 (2012).
 50.
Bassett, D. S. & Sporns, O. Network neuroscience. Nat. Neurosci. 20, 353–364 (2017).
 51.
Sompolinsky, H., Crisanti, A. & Sommers, H. J. Chaos in random neural networks. Phys. Rev. Lett. 61, 259–262 (1988).
 52.
Saito, H., Katahira, K., Okanoya, K. & Okada, M. Statistical mechanics of structural and temporal credit assignment effects on learning in neural networks. Phys. Rev. E 83, 051125 (2011).
 53.
Frémaux, N. & Gerstner, W. Neuromodulated spiketimingdependent plasticity, and theory of threefactor learning rules. Front. Neural Circuits 9, 85 (2016).
 54.
Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by backpropagating errors. Nature 323, 533–536 (1986).
Acknowledgements
We thank the members of the Vogels lab (particularly E. J. Agnes, R. P. Costa, W. F. Podlaski, and F. Zenke) for their insightful comments and Y. T. Kimura for creating the monkey illustration. We also thank O. Barak, T. E. J. Behrens, R. Bogacz, M. Jazayeri, and L. Susman for their helpful comments. Our work was supported by grants from the Wellcome Trust (T.P.V. and J.P.S. through WT100000, and G.H. through 202111/Z/16/Z) and the Engineering and Physical Sciences Research Council through the Life Sciences Interface Doctoral Training Centre at the University of Oxford (EP/F500394/1 to J.P.S.).
Author information
Affiliations
Contributions
J.P.S., G.H., and T.P.V. conceived the study and developed the model. J.P.S. performed simulations for Figs. 1–5 and Supplementary Figs. 1–4, and J.P.S. and G.H. performed simulations for Figs. 6–8 and Supplementary Figs. 5–8. J.P.S. analyzed the results, produced the figures, and wrote the first draft of the manuscript. J.P.S., M.A.P., G.H., and T.P.V. discussed and iterated on the analysis and its results, and all authors also revised the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Integrated supplementary information
Supplementary Figure 1 Further effects of neuronspecific gain modulation.
a, Changes in the largest real part in the spectrum of W×diag(g) that result from 10 different training sessions (see our simulation details). Although this change seems substantial, the resulting firingrate activity does not change dramatically. (For example, see b and the far left panel of Fig. 2b) b, Pearson correlation matrices of the firing rates for all pairs of neurons with (left) all gains set to 1 and (centre and right) two (of the 10) trained gain patterns for the task in Fig. 1d. The order of neurons is the same in all three matrices. Training does not result in a substantial reorganization in Pearson correlations between pairs of neurons. c, Histogram of the Pearson correlation coefficients between the 45 pairs of the 10 trained gain patterns for the task in Fig. 1d. d, Mean error between the network output with white Gaussian noise added to the initial condition x_{0} and the network output without noise added to x_{0} for various signaltonoise ratios for 1,000 different samples of such noise. We plot results with all gains set to 1 (blue) and the 10 trained gain patterns (red) for the task in Fig. 1d. Shading indicates 1 standard deviation. e, Left: The mean Pearson correlation coefficient between the neuronal firing rates and the target increases after training. (We show 10 training sessions, and we use a paired Wilcoxon signed rank onesided test to generate a pvalue of p ≈ 0.002.) Bottom right: Example change in Pearson correlation coefficients between the 200 neurons’ firing rates and the target after training for the trial in gray in the left panel. Top right: Example of a substantial change in the dynamics of one neuron after training. f, Box plots of the errors after training independently on 10 different target movements using backpropagation for four different scenarios. In these examples, we train either the neuronal gains, the initial condition, the recurrent synaptic weight matrix, or a rank1 perturbation of the synaptic weight matrix. See our full simulation descriptions in the supplementary material). The dashed black line is the mean error over the 10 target movements before training. (Center lines indicate median errors, boxes indicate 25th to 75th percentiles, whiskers indicate ± 1.5× the interquartile range, and dots indicate training sessions whose error lies outside the whiskers
Supplementary Figure 2 Additional results for grouped gain modulation.
a, Mean error over 10 training sessions (where shading indicates one standard deviation) using (left) random and (right) specialized groupings for 2, 10, 20, and 200 (i.e., neuronspecific) groups (see our simulation details). The target output is the same as in Fig. 1. b, Relative improvement in performance compared with neuronspecific modulation for each of 5 movements when using specialized groups shared across all (squares) or for each (circles) of the 5 movements using either 10 (blue) or 20 (black) groups. A value of 2 implies that the error is 2 times smaller after training compared to neuronspecific modulation. We indicate the performance of neuronspecific modulation using the red line. c, Mean error over 10 training sessions (where shading indicates one standard deviation) when learning 5 movements using either 20 specialized groups (shared across all 5 movements), 20 random groups, or neuronspecific modulation. d, Mean error over 10 training sessions when learning 10 novel movements using the specialized grouping (with 20 groups) shared across the 5 previously trained movements from c. e, The firing rates of 50 inhibitory and 50 excitatory neurons for each of the three different networks sizes. f, The curves give the mean error over 10 training sessions and across the 3 networks for each of 5 targets. The circles represent the mean error for each network, and the different colours indicate each of 5 different target outputs. (See our simulation details and our full simulation descriptions in the supplementary material.) g, Outputs for all five targets from the trial that produces the median error for the 400neuron network for the cases of 10 and 20 groups. h, Box plots (in blue) of the minimum error after training for different numbers of groups and the 3 different network sizes. (These are the same data that we plotted in f.) We also include box plots (in red) for the minimum number of iterations required before the error is within 1% of the minimum error. (Center lines indicate median errors, boxes indicate 25th to 75th percentiles, whiskers indicate ± 1.5× the interquartile range, and dots indicate training sessions whose error (or number of iterations) lies outside the whiskers.)
Supplementary Figure 3 Additional results for gain patterns providing motor primitives.
a, The resulting distribution of gains from training independently on each of 100 target outputs (see our simulation details). The distribution of the gain patterns resembles a normal distribution (blue curve) with the same mean and variance as those in Fig. 1e. b, Each output from the 100 trained gain patterns. c, Outputs of 100 randomlygenerated gain patterns from the distribution in a. (See our simulation details and our full simulation descriptions in the supplementary material for further details.) The outputs are substantially more homogeneous than those in b and likely would not constitute a good library for movement generation. d, The same plot as in Fig. 4d, but for up to l = 50 library elements. e, The distributions of errors across 100 different libraries for (left) l = 5 and (right) l = 20. (Note the difference in horizontalaxis scales in the two plots.) f, The error between the output and the fit from d with a different vertical axis scale. g, The same plot as in Fig. 4c, but for l = 1,..., 50 and with extended axes. Each point represents the 50thsmallest error between the output and the fit across 100 novel target movements for each of 100 randomlygenerated combinations of l library elements. We show the identity line in gray. h, The same as in g, but each point represents the 50thsmallest error between the output and the fit across the 100 libraries for each of the 100 novel target movements. We plot these data in the square [0, 1] × [0, 1] and for l = 1,..., 20. i, For the data in g, we plot the Pearson correlation coefficient between the output and the fit errors over the 100 randomlygenerated libraries for each number of library elements (up to l = 50). j, For the data in h, we plot the Pearson correlation coefficient between the output and the fit errors over the 100 novel target movements for each number of library elements (up to l = 50)
Supplementary Figure 4 Gain patterns as motor primitives with r_{0} = 5 Hz.
a, Example target (gray), fit (dashed red), and output (orange) that produces the 50thsmallest output error over 100 randomlygenerated combinations (see our simulation details for a description of the generation process) of l library elements using l = 2, l = 4, l = 8, and l = 16. b, Fit error versus the output error for 100 randomlygenerated combinations of l library elements for l = 1,..., 20. Each point represents the 50thsmallest error between the output and the fit across 100 novel target movements. We show the identity line in gray. c, For the data in b, we plot the Pearson correlation coefficient between the output and the fit errors over the 100 randomlygenerated combinations of library elements for each number of library elements (up to l = 50). d, The same as c, but for data corresponding to the 50thsmallest error for each of the 100 novel target movements, rather than for each randomlygenerated combination of library elements (up to l = 50) (see our simulation details). Compare c and d of this figure with i and j in Supplementary Fig. 3
Supplementary Figure 5 Additional results for controlling movement speeds through gain modulation.
a, Mean error over 10 training sessions for each of 10 different movements when learning gain patterns for slowmovement variants using our rewardbased learning rule (see our simulation details). b, Mean error over 10 training sessions for the same 10 movements when instead learning gain patterns for slowmovement variants using a backpropagation algorithm (see our simulation details). c, Distribution of gains for the slowmovement variants across all training sessions using our rewardbased learning rule. d, Distribution of gains for the slowmovement variants across all training sessions when using backpropagation. e, Histograms of the real and imaginary parts of the eigenvalues of the linearization of equation (1) around x = 0 before and after training using our rewardbased rule for the example in Fig. 6b. f, Histograms of the real and imaginary parts of the eigenvalues of the linearization of equation (1) around x = 0 before and after training using the backpropagation algorithm for the example in Fig. 6c. g, On the left and right, respectively, we show the same outputs that we plotted in Figs. 6b and 6c, but we now add white Gaussian noise (with a signaltonoise ratio of 4 dB) to the initial condition of the neuronal activity. (See our full simulation descriptions in the supplementary material.) h, Box plot of the slowvariant errors across 10 training sessions after training for different numbers of initial conditions. (Center lines indicate median errors, boxes indicate 25th to 75th percentiles, whiskers indicate ± 1.5× the interquartile range, and dots indicate training sessions whose error lies outside the whiskers.) (See our full simulation descriptions in the supplementary material for further details.) i, Mean error during training over 10 training sessions for m = 1,..., 10 initial conditions. j, For the case of 6 initial conditions in panel (h), we plot 4 example outputs that correspond to the 5thsmallest error for the 10 training sessions. (For each simulation in this figure, we train a 400neuron network using 40 random modulatory groups (see our simulation details)
Supplementary Figure 6 Additional results for smooth control of movement speeds through gain modulation.
a, We show outputs that result from the 7 trained gain patterns from Fig. 6e (which we also reproduce here in b) for both initial conditions (see our simulation details). b, Top: We reproduce the 7 optimized gain patterns for all 40 modulatory groups when training at 7 evenly spaced speeds from Fig. 6e. We call this the ‘speed manifold’ in the main text. Bottom: We linearly interpolate between the fast and the slow gain patterns. We use this interpolation for the outputs that we show in Fig. 6f
Supplementary Figure 7 Additional results for learning gainpattern primitives to control movement shape and speed.
We plot histograms of the errors over the 100 target movements from Fig. 8 at both fast (blue) and slow (orange) speeds (see our simulation details)
Supplementary Figure 8 Learning slowmovement variants when scaling both the amplitude and duration of target movements.
We can perform the same task as the one that we showed in Fig. 6b when we also scale the amplitude of the slowvariant target movement by the factor 1/25 (see the dashed curves). Scaling the slowvariant target movement by this factor corresponds to the same movement, but it lasts 5 times longer (see “Creating target muscle activity in Methods). We also reproduce the results from the top panel of Fig. 6b (solid curves) for comparison. During training, we reduce the errors from approximately 128 to 0.9 (i.e., a 99.3% reduction) for the amplitudescaled task and from approximately 1.22 to 0.02 (i.e., a 98.36% reduction) for the amplitudefixed task. The error for the amplitudescaled task is larger than that for the amplitudefixed task, because we scale the error by the total sum of squared errors of the target. (See equation (7) for the definition of error that we use.)
Supplementary information
Rights and permissions
About this article
Cite this article
Stroud, J.P., Porter, M.A., Hennequin, G. et al. Motor primitives in space and time via targeted gain modulation in cortical networks. Nat Neurosci 21, 1774–1783 (2018). https://doi.org/10.1038/s4159301802760
Received:
Accepted:
Published:
Issue Date:
Further reading

Towards a functional connectome in Drosophila
Journal of Neurogenetics (2020)

On Primitives in Motor Control
Motor Control (2020)

Neuronal mechanisms for sequential activation of memory items: Dynamics and reliability
PLOS ONE (2020)

Gain control in the sensorimotor system
Current Opinion in Physiology (2019)

Brian 2, an intuitive and efficient neural simulator
eLife (2019)