Animals readily execute learned behaviors in a consistent manner over long periods of time, and yet no equally stable neural correlate has been demonstrated. How does the cortex achieve this stable control? Using the sensorimotor system as a model of cortical processing, we investigated the hypothesis that the dynamics of neural latent activity, which captures the dominant co-variation patterns within the neural population, must be preserved across time. We recorded from populations of neurons in premotor, primary motor and somatosensory cortices as monkeys performed a reaching task, for up to 2 years. Intriguingly, despite a steady turnover in the recorded neurons, the low-dimensional latent dynamics remained stable. The stability allowed reliable decoding of behavioral features for the entire timespan, while fixed decoders based directly on the recorded neural activity degraded substantially. We posit that stable latent cortical dynamics within the manifold are the fundamental building blocks underlying consistent behavioral execution.
Subscribe to Journal
Get full journal access for 1 year
only $17.42 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The datasets analyzed for this manuscript will be shared upon reasonable request.
All analyses were implemented using custom Matlab (The Mathworks Inc.) code. Code to replicate the main results will be shared upon reasonable request.
Gallego, J. A., Perich, M. G., Miller, L. E. & Solla, S. A. Neural manifolds for the control of movement. Neuron 94, 978–984 (2017).
Elsayed, G. F. & Cunningham, J. P. Structure in neural population recordings: an expected byproduct of simpler phenomena? Nat. Neurosci. 20, 1310–1318 (2017).
Sadtler, P. T. et al. Neural constraints on learning. Nature 512, 423–426 (2014).
Stopfer, M., Jayaraman, V. & Laurent, G. Intensity versus identity coding in an olfactory system. Neuron 39, 991–1004 (2003).
Gallego, J. A. et al. Cortical population activity within a preserved neural manifold underlies multiple motor behaviors. Nat. Commun. 9, 4233 (2018).
Pandarinath, C. et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nat. Methods 15, 805–815 (2018).
Trautmann, E. M. et al. Accurate estimation of neural population dynamics without spike sorting. Neuron 103, 292–308 (2019).
Gao, P. et al. A theory of multineuronal dimensionality, dynamics and measurement. Preprint at bioRxiv https://doi.org/10.1101/214262 (2017).
Cunningham, J. P. & Yu, B. M. Dimensionality reduction for large-scale neural recordings. Nat. Neurosci. 17, 1500–1509 (2014).
Dickey, A. S., Suminski, A., Amit, Y. & Hatsopoulos, N. G. Single-unit stability using chronically implanted multielectrode arrays. J. Neurophysiol. 102, 1331–1339 (2009).
Stevenson, I. H. et al. Statistical assessment of the stability of neural movement representations. J. Neurophysiol. 106, 764–774 (2011).
Dekleva, B. M., Kording, K. P. & Miller, L. E. Single reach plans in dorsal premotor cortex during a two-target task. Nat. Commun. 9, 3556 (2018).
Santhanam, G. et al. Factor-analysis methods for higher-performance neural prostheses. J. Neurophysiol. 102, 1315–1330 (2009).
Rathelot, J.-A. & Strick, P. L. Muscle representation in the macaque motor cortex: an anatomical perspective. Proc. Natl Acad. Sci. USA 103, 8257–8262 (2006).
Morrow, M. M. & Miller, L. E. Prediction of muscle activity by populations of sequentially recorded primary motor cortex neurons. J. Neurophysiol. 89, 2279–2288 (2003).
Churchland, M. M. et al. Neural population dynamics during reaching. Nature 487, 51–56 (2012).
Cherian, A., Fernandes, H. L. & Miller, L. E. Primary motor cortical discharge during force field adaptation reflects muscle-like dynamics. J. Neurophysiol. 110, 768–783 (2013).
London, B. M. & Miller, L. E. Responses of somatosensory area 2 neurons to actively and passively generated limb movements. J. Neurophysiol. 109, 1505–1513 (2013).
Prud’homme, M. J. & Kalaska, J. F. Proprioceptive activity in primate primary somatosensory cortex during active arm reaching movements. J. Neurophysiol. 72, 2280–2301 (1994).
Sainburg, R. L., Ghilardi, M. F., Poizner, H. & Ghez, C. Control of limb dynamics in normal subjects and patients without proprioception. J. Neurophysiol. 73, 820–835 (1995).
Sussillo, D., Churchland, M. M., Kaufman, M. T. & Shenoy, K. V. A neural network that finds a naturalistic solution for the production of muscle activity. Nat. Neurosci. 18, 1025–1033 (2015).
Perich, M. G., Gallego, J. A. & Miller, L. E. A neural population mechanism for rapid learning. Neuron 100, 964–976.e7 (2018).
Santhanam, G., Ryu, S. I., Yu, B. M., Afshar, A. & Shenoy, K. V. A high-performance brain–computer interface. Nature 442, 195–198 (2006).
Ethier, C., Oby, E. R., Bauman, M. J. & Miller, L. E. Restoration of grasp following paralysis through brain-controlled stimulation of muscles. Nature 485, 368–371 (2012).
Collinger, J. L. et al. High-performance neuroprosthetic control by an individual with tetraplegia. The Lancet 381, 557–564 (2013).
Sussillo, D., Stavisky, S. D., Kao, J. C., Ryu, S. I. & Shenoy, K. V. Making brain–machine interfaces robust to future neural variability. Nat. Commun. 7, 13749 (2016).
Churchland, M. M. Neural variability in premotor cortex provides a signature of motor preparation. J. Neurosci. 26, 3697–3712 (2006).
Scott, S. H. Optimal feedback control and the neural basis of volitional motor control. Nat. Rev. Neurosci. 5, 532–545 (2004).
Rokni, U., Richardson, A. G., Bizzi, E. & Seung, H. S. Motor learning with unstable neural representations. Neuron 54, 653–666 (2007).
Chestek, C. A. et al. Single-neuron stability during repeated reaching in macaque premotor cortex. J. Neurosci. 27, 10742–10750 (2007).
Nonnenmacher, M., Turaga, S. C. & Macke, J. H. Extracting low-dimensional dynamics from multiple large-scale neural population recordings by learning to predict correlations. Adv. Neural Inf. Process. Syst. 30, 5702–5712 (2017).
Stevenson, I. H., Rebesco, J. M., Miller, L. E. & Körding, K. P. Inferring functional connections between neurons. Curr. Opin. Neurobiol. 18, 582–588 (2008).
Wärnberg, E. & Kumar, A. Perturbing low dimensional activity manifolds in spiking neuronal networks. PLoS Comput. Biol. 15, e1007074 (2019).
Mastrogiuseppe, F. & Ostojic, S. Linking connectivity, dynamics, and computations in low-rank recurrent neural networks. Neuron 99, 609–623.e29 (2018).
Marshel, J. H. et al. Cortical layer-specific critical dynamics triggering perception. Science 365, eaaw5202 (2019).
Oby, E. R. et al. New neural activity patterns emerge with long-term learning. Proc. Natl Acad. Sci. USA 116, 15210–15215 (2019).
Miri, A. et al. Behaviorally selective engagement of short-latency effector pathways by motor cortex. Neuron 95, 683–696.e11 (2017).
Bensmaia, S. J. & Miller, L. E. Restoring sensorimotor function through intracortical interfaces: progress and looming challenges. Nat. Rev. Neurosci. 15, 313–325 (2014).
Chestek, C. A. et al. Long-term stability of neural prosthetic control signals from silicon cortical arrays in rhesus macaque motor cortex. J. Neural Eng. 8, 045005 (2011).
Wu, W. & Hatsopoulos, N. G. Real-time decoding of nonstationary neural activity in motor cortex. IEEE Trans. Neural Syst. Rehabil. Eng. 16, 213–222 (2008).
Stavisky, S. D., Kao, J. C., Nuyujukian, P., Ryu, S. I. & Shenoy, K. V. A high performing brain–machine interface driven by low-frequency local field potentials alone and together with spikes. J. Neural Eng. 12, 036009 (2015).
Flint, R. D., Scheid, M. R., Wright, Z. A., Solla, S. A. & Slutzky, M. W. Long-Term Stability of motor cortical activity: implications for brain machine interfaces and optimal feedback control. J. Neurosci. 36, 3623–3632 (2016).
Orsborn, A. L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393 (2014).
Dyer, E. L. et al. A cryptography-based approach for movement decoding. Nat. Biomed. Eng. 1, 967–976 (2017).
Farshchian, A. et al. Adversarial domain adaptation for stable brain–machine interfaces. in International Conference on Learning Representations (ICLR) https://openreview.net/forum?id=Hyx6Bi0qYm (2019).
Kao, J. C., Ryu, S. I. & Shenoy, K. V. Leveraging neural dynamics to extend functional lifetime of brain–machine interfaces. Sci. Rep. 7, 7395 (2017).
Lara, A. H., Cunningham, J. P. & Churchland, M. M. Different population dynamics in the supplementary motor area and motor cortex during reaching. Nat. Commun. 9, 2754 (2018).
Remington, E. D., Narain, D., Hosseini, E. A. & Jazayeri, M. Flexible sensorimotor computations through rapid reconfiguration of cortical dynamics. Neuron 98, 1005–1019.e5 (2018).
Kobak, D. et al. Demixed principal component analysis of neural population data. eLife 5, e10989 (2016).
Pandarinath, C. et al. Neural population dynamics in human motor cortex during movements in people with ALS. eLife 4, e07436 (2015).
Georgopoulos, A., Kalaska, J., Caminiti, R. & Massey, J. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982).
Moran, D. W. & Schwartz, A. B. Motor cortical representation of speed and direction during reaching. J. Neurophysiol. 82, 2676–2692 (1999).
Perich, M. G. & Miller, L. E. Altered tuning in primary motor cortex does not account for behavioral adaptation during force field learning. Exp. Brain Res. 235, 2689–2704 (2017).
Yu, B. M. et al. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. J. Neurophysiol. 102, 614–635 (2009).
Cisek, P. & Kalaska, J. F. Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45, 801–814 (2005).
Bach, F. R. & Jordan, M. I. Kernel independent component analysis. J. Machine Learn. Res. 3, 1–48 (2002).
Glaser, J. I. et al. Machine learning for neural decoding. Preprint at arXiv https://arxiv.org/abs/1708.00909 (2017).
This work was supported in part by: grant no. FP7-PEOPLE-2013-IOF-627384 from the Commission of the European Union and grant no. 2017-T2/TIC-5263 from the Community of Madrid to J.A.G.; grant no. F31-NS092356 from the National Institute of Neurological Disorder and Stroke and grant no. T32-HD07418 from the National Institute of Child Health and Human Development to M.G.P.; grant no. DGE-1324585 from the National Science Foundation to R.H.C.; and grant nos. NS095251 to L.E.M. and NS053603 to S.A.S. and L.E.M. from the National Institute of Neurological Disorder and Stroke.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
(a-f) Correlation between direction-matched single trial X and Y hand velocities across all pairs of days (single dots: individual trials; lines: linear fits) from Monkey CL (a), Monkey J (b), Monkey M (c), Monkey T (d), Monkey P (e), and Monkey H (f). The inset in (c) shows X and Y hand trajectories for three example sessions. Trajectories are color coded by target as in Fig. 2.
Extended Data Fig. 2 Additional data: example neural activity during reaching on two days from Monkey CR.
Each row shows the firing rates on a different electrode for Day 27 (left column) and Day 43 (right column). Each color represents a different sorted neuron. The eight plots arranged in a circular manner show the firing rate as a function of time during a reach to each of the eight targets, aligned on movement onset and averaged across all trials to the same target. The inset in the top left of each panel shows the average waveform of each sorted neuron; the inset at the top right shows the ISI distribution for each sorted neuron. Inset scale bars: horizontal, 400 µs; vertical, 200 µV.
(a) We manually spike-sorted the neural recordings from Monkeys C, M, and T to establish whether the same neurons were recorded across days (Methods; Extended Data Fig. 2 shows example neurons). Plots show the average action potential waveform of example sorted neurons for two datasets: Day 27 and Day 43 from Monkey CL. Note the large apparent turnover after 15 days. Right insets: example action potential waveforms and inter-spike interval (ISI) histograms for two neurons that were matched across days. (b) To quantify the turnover effect, we tracked both firing rate statistics and waveform shape of each neuron; these figures show the percentage of individual sorted M1 neurons that were matched across pairs of days based on action potential waveforms and inter-spike interval (ISI) histograms. Data from Monkey CL (top), Monkey CR (middle; inset highlights the first 35 days), and Monkey M (bottom; inset highlights the first 50 days). (c) Percentage of individual sorted PMd neurons that were matched across pairs of days as in (b). Data from Monkey CL (top), Monkey M (middle), and Monkey T (bottom).
For each implant: change in mean firing rate (top plot), modulation depth (middle plot), and preferred direction (bottom plot) of standard cosine tuning fits to multiunit activity across all pairs of days. Line and shaded areas: mean ± s.e.m. Plots are grouped by implant and brain area (M1: left; PMd: middle; S1: right). Error bars: 95% confidence interval of linear fit. n: number of across-day comparisons.
(a) Correlation of the aligned (CCs; red) and unaligned (Pearson’s r; orange) M1 latent dynamics averaged over the top four neural modes across all pairs of days from Monkey CL using a 6-D manifold (single dots: pairs of days; lines: linear fits). (b) Normalized similarity of the aligned and unaligned M1 latent dynamics in the 6-D neural manifold for Monkey CL. (c) Mean and s.e.m. for normalized similarity distributions as shown in (b), for all four M1 implants for 6, 8, 10, and 12-D manifolds. The 10-D data presented here summarizes the distributions shown in Fig. 4. The significance of the separation between aligned and unaligned distributions held regardless of the choice of neural manifold dimensionality. N values are the same as for the corresponding distributions in Fig. 4. (d) Correlation (CCs) of the M1 latent dynamics averaged over the top four neural modes across all pairs of days from Monkey CL using sorted neurons rather than multiunit activity (single dots: pairs of days; lines: linear fits). (e) Normalized similarity of the aligned and unaligned M1 latent dynamics in the 10-D manifold obtained using sorted neurons for Monkeys CL, CR, and M. Error bars: mean ± s.d. n: number of across-day comparisons.
(a) Predictive accuracy when decoding hand velocity for all pairs of days from Monkey CL using the unaligned latent dynamics as inputs instead of the multiunit activity used in Fig. 5. (b) Predictive accuracy when using as inputs the latent dynamics within-day and across-day both before and after alignment, for Monkeys CL, CR, and M. *** denotes p < 0.001, two-sided Wilcoxon rank-sum test. Error bars: mean ± s.d. n: number of across-day comparisons.
(a) Simulation showing that movement tuning does not account for unchanging latent dynamics, as in Fig. 6a–d. Latent dynamics from Day 1 (purple curves) are nonlinearly but smoothly transformed into latent dynamics of Day n (pink curves). The latent dynamics are shown as projections onto the four leading neural modes. (b) This transformation preserves neural firing statistics across the population. N=88 neurons; box plot shows median and 25th/75th percentiles, whiskers show range. (c,d) The statistics of preferred directions are also well-preserved across the population. Panels (a-d) present data pooled across all sessions from Monkey CL. (e) As an additional control, we used the TME method to generate simulated population neural activity that preserved the covariance across neurons and conditions (targets), while the covariance over time (dynamics) was not constrained to be preserved. Example data from Monkey CR. Legend: Cov. T: covariance over time; Cov. N: covariance across neurons; Cov. Tgt: covariance across targets. (f) Distribution of the averaged top four CCs between the simulated data and the recorded data for M1 recordings from three monkeys (grey). The distribution for the within-day averaged top four CCs for the recorded data (black) is shown for reference. ***: p< 0.001, two-sided Wilcoxon rank-sum test. Error bars: mean ± s.d. n: number of within-session comparisons.
Extended Data Fig. 8 Additional control data: Stable latent dynamics are not a byproduct of single neuron tuning to movement.
(a) Contribution to the latent dynamics from tuned vs untuned neurons: The neural population was divided into two subpopulations based on the quality of a cosine fit to the activity of each neuron. The average activity in the neural manifold for reaches to each of the eight targets are shown for one example session; one data point per reach. The clustering by target direction observed in the full population (left) was preserved for the tuned subpopulation (middle) but not for the untuned subpopulation (right). (b) Distribution of the averaged top four CCs between the tuned subpopulation and the full population (red), and between the untuned subpopulation and the full population (blue) for all M1 sessions. The dynamics of the untuned population could be well aligned with the dynamics of the full population. Data pooled over all sessions from Monkey CL. (c) A static model based on movement tuning properties of individual neurons represents reaches to each target with one data point per trial and results in target-specific clusters that can be aligned. (d) Left: each point represents a reach to one of the eight targets (color code in inset) on Day 1 (closed circles) and Day n (open squares). Target specificity is mostly lost when these points are projected onto their respective manifolds. Right: after alignment, similar target-specific structure is present for both days. (e) Pairwise comparisons of the CCs after projecting the latent dynamics onto the manifold axes found by aligning the clusters (vertical axis) and onto the manifold axes found by aligning the latent dynamics (horizontal axis). Data shown for the top six neural modes (see legend for color code). Each dot represents one session comparison. All dots lie below the diagonal (dashed grey), indicating that aligning the statistics of the population activity based on target-specific clusters does not reach the CC values obtained by aligning the latent dynamics. (f) Canonical correlation values were significantly lower when the static clusters as opposed to the latent dynamics were aligned, illustrating the importance of the precise temporal dynamics for accurate alignment. (g) Consequently, across-day decoding was notably worse when aligning the static clusters.***: p < 0.001, two-sided Wilcoxon rank-sum test. Error bars: mean ± s.d. n: number of across-day comparisons.
(a) Example mean neural firing rates for 51 PMd multiunits recorded on Day 27 and Day 43 from Monkey CL (top; each multiunit is shown in a different row) and corresponding hand velocity (bottom). Each column represents the average of all trials to each of the eight reach directions (indicated by the arrows above each column). Data was recorded during the pre-movement planning and the transition to movement; hand velocities are thus largely zero. Note the substantial changes in the planning activity of the recorded PMd multiunits across days. Velocity scale bars: horizontal, 300 ms; vertical, 10 cm/s. (b) Correlation of the aligned (CCs; red) and unaligned (Pearson’s r ; orange) PMd latent dynamics averaged over the top four neural modes across all pairs of days from Monkey M (single dots: pairs of days; lines: linear fits). (c) Same as (b) for Monkey T. (d) Classification accuracy for classifiers trained and tested on all different pairs of days for Monkey M (left). (e) Same as (d) for Monkey T.
(a) Example mean neural firing rates aligned to movement onset for 65 S1 multiunits recorded on Day 1 and Day 29 from Monkey P (top; each multiunit shown in a different row) and corresponding hand velocity (bottom). Each column represents the average of all trials to each of the eight reach directions (indicated by the arrows above each column). Velocity scale bars: horizontal, 300 ms; vertical, 10 cm/s. (b) Correlation of the aligned (CCs; red) and unaligned (Pearson’s r; orange) S1 latent dynamics averaged over the top four neural modes across all pairs of days from Monkey H (single dots: pairs of days; lines: linear fits). (c) Predictive accuracy for decoders trained and tested on all different pairs of days for Monkey P. (d) Same as (c) for Monkey H.
About this article
Cite this article
Gallego, J.A., Perich, M.G., Chowdhury, R.H. et al. Long-term stability of cortical population dynamics underlying consistent behavior. Nat Neurosci 23, 260–270 (2020). https://doi.org/10.1038/s41593-019-0555-4
Classifying Intracortical Brain-Machine Interface Signal Disruptions Based on System Performance and Applicable Compensatory Strategies: A Review
Frontiers in Neurorobotics (2020)
Annual Review of Neuroscience (2020)
Cortical Observation by Synchronous Multifocal Optical Sampling Reveals Widespread Population Encoding of Actions
Current Opinion in Neurobiology (2020)