Abstract
Our knowledge about neuronal activity in the sensorimotor cortex relies primarily on stereotyped movements that are strictly controlled in experimental settings. It remains unclear how results can be carried over to less constrained behavior like that of freely moving subjects. Toward this goal, we developed a selfpaced behavioral paradigm that encouraged rats to engage in different movement types. We employed bilateral electrophysiological recordings across the entire sensorimotor cortex and simultaneous paw tracking. These techniques revealed behavioral coupling of neurons with lateralization and an anterior–posterior gradient from the premotor to the primary sensory cortex. The structure of population activity patterns was conserved across animals despite the severe undersampling of the total number of neurons and variations in electrode positions across individuals. We demonstrated crosssubject and crosssession generalization in a decoding task through alignments of lowdimensional neural manifolds, providing evidence of a conserved neuronal code.
Similar content being viewed by others
Introduction
Humans and animals are capable of generating a vast array of behaviors. This feature is dependent on the brain’s ability to generate a wide repertoire of neural activity patterns, which may rely on subsets of general motifs^{1}. Experimental, computational, and theoretical work has identified the rich underlying structures within neural populations regarding movement control, decisionmaking, and memory tasks^{2}. Similarities in population structures across different modalities such as speech and arm movements^{3}, as well as the relevance of populationlevel phenomena to learning^{4}, hint at the existence of general principles that could be shared across subjects. For simple, constrained behavior such as running on a linear track, population structures in some brain regions such as the hippocampus seem to be conserved, even across subjects^{5}. Similarities in neural population structures have not yet been shown for freely roaming animals and various naturally occurring behaviors. Whether population structures are sufficiently conserved across subjects to allow for the crosssubject decoding of behavioral categories remains an open question in systems neuroscience. This question has great implications for neuroprosthetic approaches, among other research topics. Such conservation of neural structures would allow for a shorter adaptation or finetuning phase of BrainMachineInterface (BCI) systems from one subject to another as opposed to training the system from scratch. We addressed this question with nonlinear mapping applied to electrophysiological recordings across the entire bilateral sensorimotor cortex of the rat. The neural trajectories of dynamical systems have been suggested as a method to understand neural activity^{4,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21}. Therefore, we built on Laplacian Eigenmaps (LEMs)^{5,22}, which map highdimensional data via the data’s affinity to a lowdimensional manifold. When affinities are defined according to neuronal population activity, they can be used as tools to visualize structures and relationships among population activities at different time points of a recording session in a lowdimensional space. This can potentially reveal conserved structures across sessions and animals^{5}. To investigate the degree to which lowdimensional structures are conserved, it is necessary to involve several types of behavior. In principle, it is possible to train animals in different tasks, but this has several limitations: (1) training animals is timeconsuming, especially if multiple behaviors are involved; (2) the trained behavior often results in stereotyped movements and (due to the plasticity of the mammalian brain) corresponding changes in neuronal representations; and (3) frequent transitions between behaviors are not feasible. Furthermore, spontaneous movements influence neuronal activity, even in wellcontrolled tasks^{23}. Therefore, we refrained from controlling the behavior from the start, instead allowing the rats to roam freely in a Plexiglas box. Consequently, the animals showed a wide range of naturalistic behavior, such as rearing, grooming, turning, stepping, drinking, and resting. To verify this approach, we first compared neuronal activity with previously reported results from more constrained behaviors by focusing on step and swinglike paw movements. This study confirmed that the quality of information conveyed by our recorded data was comparable to that found in conventionally controlled settings. In addition, we reported a strong anterior–posterior gradient in the lateralization of forelimb representations from the premotor cortex to the primary sensory cortex. This gradient emphasizes the strong involvement of more posterior regions in the encoding of steplike behavior. After this validation, we focused on analyzing the population code for more complex behaviors. We conducted a normal withinsession decoding experiment to show that the neuronal code contains enough information about the behavior classes. Across sessions, the signals of individual neurons were not comparable since neurons typically cannot be traced over multiple days. Across subjects, even the electrode positions varied. However, we found evidence that the signal from the population of neurons shared a common structure across sessions and even across subjects. In particular, decoding behavioral categories from the neuronal population activity was possible across different subjects.
Results
Rats moved unconstrained in a rectangular arena and conducted movements in different behavioral categories (i.e., stepping, turning, drinking, grooming, and rearing) while searching for water drops, which a robot arm positioned under mesh occasionally delivered (Fig. 1a). We recorded neuronal activities using electrodes that covered the sensorimotor cortex over both hemispheres (Fig. 1b). Two cameras videotaped the behavior of the rats for simultaneous 3D tracking. Recording sessions (n = 106 in total) were distributed over three months and varied between 30 and 60 min (μ = 36.1 min, σ = 5.2 min). In total, we identified 3723 singleunits (μ = 35, σ = 21 across sessions) that we used for further analysis: 734, 896, and 231 in the left M2, M1, and S1, respectively, and 435, 796, and 631 in the right M2, M1, and S1, respectively^{24}.
We focused on steplike behavior to extract behavioral components from the movements. To extract the steps, we binarized the movements of the paws into swing (moving) and stance (not moving) according to a horizontal velocity threshold (0.03 mm/ms). With each paw, rats performed one step per second on average (μ = 1.22, σ = 0.29). The average percentage of time spent in the stance phase across rats was 71%, σ = 17%.
The strongest paw coupling in contralateral S1
Since classical methods such as peristimulus time histograms (PSTHs) are not applicable to behavior without a trial structure, we computed spiketriggered averages to investigate the relationship between neuronal activity and unconstrained movements^{25}. We defined the spiketriggered average paw swing–stance status (STAPSSS) as a rough measure of the coupling of individual neurons to paw movements. For each neuron and each paw, we calculated the STAPSSS by averaging the swing–stance status in the period ±1 s around the spikes (Fig. 1c). For statistical control, we randomly shifted the spike train 1000 times to calculate 1000 control STAPSSS waveforms. Bootstrapping via temporal shifting preserves autocorrelations in time series and, therefore, helps to exclude false positives arising solely from autocorrelations in paw movements. We considered the STAPSSSs to be significant if their standard deviation over time exceeded the 0.99 quantile standard deviation of the control STAPSSS waveforms. Only neurons that spiked more systematically than expected by chance in relation to movement parameters could pass this test. Significantly coupled neurons were characterized by clear peaks in the STAPSSS (Fig. 1d and Supplementary Fig. 1). In total, 54% (2029/3723) of all neurons were significantly coupled to at least one paw. These were 47% (546/1169) of all neurons in M2, compared to 53% (906/1692) in M1 and 67% (577/862) in S1.
To take into account the strength of coupling, we defined a continuous measure for paw coupling as the ratio of the STAPSSS’s standard deviation and the control standard deviation (>1 for significant neurons). Using this quotient as a dependent variable, we calculated threeway ANOVAs (with hemisphere, area, and rat as factors) for all four paws separately (detailed results in Supplementary Table 2). In summary, for all four paws, we found a stronger coupling on the contralateral side (p = 0.04), which suggests lateralization during locomotion. The coupling increased from anterior to posterior areas (p < 1e − 11). For all four paws, the highest mean coupling was localized in contralateral S1 (Fig. 1e). In three out of the four paws, the interaction between the area and hemisphere was also significant, that is, the differences between the contralateral and ipsilateral hemisphere increased from anterior to posterior areas (p = 0.02). To further investigate the difference in magnitude between contralateral and ipsilateral paw coupling, we defined contralateral bias as the ratio between the coupling of the contralateral and ipsilateral paws: b = c_{r}/c_{l} for lefthemispheric neurons and b = c_{l}/c_{r} for righthemispheric neurons (b ≈ 1 for nonbiased neurons), with bias denoted as b, coupling as c, the right paw as r, and the left paw as l. We calculated this bias separately for the front and hind paws. A twoway ANOVA on the contralateral bias of individual neurons revealed a significant effect of the brain area for the front paws (F_{2,3715} = 44.66, p < 1e − 19) and the hind paws (F_{2,3715} = 54.56, p < 1e − 23). This confirmed that single neurons had a larger contralateral bias from anterior to posterior areas for the front and hind paws (Supplementary Fig. 2).
To further investigate the temporal relationship between neuronal activity in different motor areas and paw movements, we quantified the offset between each movement peak (the STAPSSS peak) and spike. We found that for all four paws, the offset for neurons in S1 tended to be more negative (i.e., the spike followed movement) than that of neurons in M1 and M2 (Supplementary Fig. 3). This effect was more pronounced for the hind paws according to an unpaired Mann–Whitney Utest between offsets from S1 and M1/M2 (front left paw Mann–Whitney U = 1157042, n_{1} = 862, n_{2} = 2861, p = 0.002 twosided; front right paw U = 1155623.5, p = 0.002; hind left paw U = 1056184.5, p < 1e − 10; hind right paw U = 1070083, p < 1e − 8). The finding that neurons in S1 tended to spike after movements, whereas neurons in M1 and M2 spiked in a closer temporal relationship to paw movements, aligns well with the idea of S1 reacting to sensory input and M1 and M2 being more involved in movement generation.
Singleunit activity allows for the decoding of paw movements within sessions
Owing to the strong paw coupling, we hypothesized that it is possible to decode the paw movements of freely moving rats from neuronal activity. To test this hypothesis, we applied feedforward neural networks to decode the swing–stance status of the right front paw posed as a twoclass classification problem. For each time point, we fed in the spike trains ±400 ms of all units in time bins of 10 ms. Deep neural networks were trained and evaluated separately for each recording session. We chose this approach because singleneuron activity does not generalize over sessions, in contrast to our populationlevel decoding approach in the following section. The mean perclass decoding accuracies were well abovechance level (μ = 71%, σ = 10%; chance level 50%). While there was no significant correlation between accuracy and train set sizes (Spearman’s ρ = 0.17, p = 0.07), we found a significant correlation between the accuracy and percentage of coupled neurons according to our STAPSSS analysis per session (Spearman’s ρ = 0.63, p < 1e − 12, Fig. 1f). This confirms that STAPSSS is a reliable measure of the correlation between neuronal activity and movement.
The structure of population activities allows the decoding of behavior
Owing to the promising decoding results of paw movements, we sought to determine whether population activity during unconstrained movements also contained information on more complex behavior. We used LEMs^{5,22} to reveal and visualize the structures in the population activities. LEM is a nonlinear dimensionalityreduction method for extracting lowdimensional manifolds in highdimensional data using spectral techniques. Owing to redundancy in the code of the cortex, organized in subpopulations of correlated units, we expected in fact to capture most of the population variance in a space with a lower dimension than the recording space. We applied LEM to the neighborhood graphs of neuronal activity vectors to visualize structures and relationships among population activities at different time points of a recording session in a lowdimensional space. Most of the resulting projections showed a clear saddlelike shape when visualized in three dimensions (Fig. 2a, 52/95, roughly 55% of session structures had a similar shape, as classified visually), although there were also randomlike population structures that differed from this general trend (14/95, about 15% of the sessions, Supplementary Fig. 4; the remaining 35% had intermediate levels of structuredness). In randomlike population structures, the time points were uniformly distributed in a sphere. Sessions with a clear saddlelike shape were characterized by a larger number of neurons that were significantly coupled to at least one paw compared to sessions with an intermediate or low level of structuredness (μ = 23.6, σ = 14.7 vs. μ = 16.6, σ = 12.3 neurons, Mann–Whitney U = 797, n_{1} = 52, n_{2} = 43, p = 0.008 twosided). To ensure that the saddlelike structures were not a simple artifact of the dimensionalityreduction method, we also performed timeshuffled, neuronshuffled, and timeshifted control reductions^{5}. These did not lead to any apparent structure (Supplementary Fig. 5).
To investigate the relationship between population structures and the corresponding behavior, we proceeded by manually labeling sessions in 500 ms snippets into six behavioral classes (stepping/paw movement, turning/head movement, drinking, grooming, rearing, and resting). These six classes included complex behaviors, recruiting the full body of the rat and the full sensorimotor cortex (Supplementary Fig. 6). We included all sessions with clear saddlelike shapes and with at least five significantly coupled neurons, which resulted in a total of 48 sessions (13 for Rat A, 16 for Rat B, 7 for Rat C, 6 for Rat D, 3 for Rat E, 3 for Rat F).
While each session contained at least some samples of each behavior, the occurrences of behaviors still differed considerably across sessions and rats (Supplementary Fig. 7). In contrast, the distributions of behaviors across the neural structures revealed clear similarities across rats, which was surprising assuming a sampling of approximately 0.005% of all neurons (ratio between the number of recorded cells and the estimated total number of cells, approximated for the area covered by the implanted electrodes by assuming a cortical thickness of 2mm and a density of 90k neurons per mm^{3}^{26}) on average in only roughly overlapping recording sites (cf. Fig. 1b). For example, the second eigenvector (here: first dimension), the socalled Fiedler vector, clearly represented the difference between movement and rest (Fig. 2b left column). For some animals, a clear distinction between more pawrelated (paw movement, rearing) and headrelated behavior (head movement, drinking) was observable in the third and fourth eigenvector (here: second/third dimension, Fig. 2b right column). Although the position of a population vector in the LEM space is univocally defined by the instantaneous activity of all its composing units, and is relatively little affected by the activity of a singleunit, there is a relationship between the overall structure emerging in the LEM space when observing the totality of recorded data and the firing of neurons with high behavioral selectivity. While population vectors cluster in space due to the similarity between neuronal representations during a specific behavior, singleunits with high selectivity for such a behavior will fire more strongly at that behavior’s cluster (Supplementary Fig. 8). Moreover, the distance in the LEM space between population vectors corresponding to two behaviors will increase with the number of units within the population that change their firing rate between the two behaviors (Supplementary Fig. 9a, b).
To quantify the separation between the neuronal representations of the six identified behaviors in the LEM space, we trained a neural network based on the first 10 dimensions of the population vectors. We chose 10 dimensions because we found the mean dimensionality in the LEM space to be μ = 8.6, σ = 1.1 (see Methods). By choosing a slightly higher value than the mean dimensionality, we added a small safety margin to ensure the inclusion of all relevant dimensions. The neural network correctly classified behaviors more frequently than by chance (mean perclass accuracy μ = 47.1%, σ = 9.6%; chance level 16.66%, Fig. 2c). The accuracies were correlated to the number of significantly coupled neurons (n = 48, Pearson’s ρ = 0.59, p < 1e − 5, Fig. 2d), the total number of units (Pearson’s ρ = 0.54, p < 1e − 4, Supplementary Fig. 10a), and the signaltonoise ratio (SNR) averaged over units (Pearson’s ρ = 0.49, p < 0.001, Supplementary Fig. 10b). Common classification mistakes consisted of confusing rearing or turning with stepping, as well as turning with drinking or resting (Fig. 2e). At the singleunit level, these behavioral classes shared, in fact, the highest number of selective units (Supplementary Fig. 9c, d) in each of the three recorded regions (Supplementary Fig. 9e). Moreover, we observed the lowest accuracy for Rat D, Rat E, and Rat F. These rats had a low mean SNR (Rats E–F, Supplementary Fig. 10b) or no electrode coverage of posterior areas (Rats D and F, cf. Fig. 1b). This last aspect made us hypothesize that more posterior regions are primarily involved in the encoding of behavioral classes. To test this hypothesis, we investigated the influence of the different sensorimotor areas on neural population structures. Thus, we conducted dimensionality reductions with equal numbers of neurons (i.e., 20 randomly chosen units) from M2, M1, or S1 as input. With this subset, we trained artificial neural networks to decode the behavioral classes with the neural activity in a given area reduced to five dimensions as input. The decoding accuracies for M1 were significantly better than those for M2 (Wilcoxon signedrank test W = 155, n = 41, p < 1e − 5 twosided) and those for S1 (W = 335, n = 42, p = 0.009). In total, the accuracies were highest in M1 for 28 out of the 48 sessions, compared to 15 for S1 and 5 for M2 (Fig. 2f, accuracies μ = 25.8%, σ = 4.9% in M2, μ = 28.6%, σ = 5.3% in M1, μ = 26.7%, σ = 5.4% in S1). The low relevance of anterior sensorimotor regions is in line with the STAPSSS results, as well as with the lower decoding accuracies in Rat D and Rat F.
A crosssession polytope comparison reveals similarities in the average encoding of behavioral classes across animals
The visual similarity between neural population structures in three dimensions (cf. Fig. 2a, b) led us to wonder whether correspondences between the full dimensional structures could be quantified. Such similarities become more apparent when reducing the extended manifolds to polytopes—highdimensional polyhedra—with vertexes defined by the average population vectors associated with the six behavioral classes (Fig. 3a). This encouraged us to systematically investigate whether population activities during unconstrained movements contained structures that were conserved across recording sessions or even across different animals. We excluded Rat F from all of the following analyses because of low recording quality, which may reflect the long delay between implantation and measurements compared to the other rats (see Supplementary Table 1).
Polytopes are useful tools to facilitate the visualization of the complex structure associated with the neuronal representations of the six identified behaviors and their reciprocal distance. To test whether such distances were preserved across sessions and animals abovechance level, we first tested whether behaviors associated with similar population vectors in one session corresponded to behaviors with similar population vectors in other sessions. For example, if in one session the population vectors during turn and step are similar to each other but dissimilar from those during rest, we wondered whether the same relationship can be found in other sessions as well. More formally, for each pair of sessions v and w and each behavioral class i, we ranked the remaining classes by the Euclidean distance between their average population vector and the average population vector of class i. If v and w had the same polytope structure, the rank associated with each behavior would be identical. We quantified the similarity between ranks across sessions with the statistic \({s}_{i}^{vw}\), defined as the number of concordant ranks, and compared its distribution with that obtained from bootstrapping (see Methods for details). We found significant similarities across sessions, both when computing distances in the highdimensional recording space (Kolmogorov–Smirnov test, p < 1e − 39) and in the reduced LEM space (p < 1e − 73) (Supplementary Fig. 11a, b). This finding confirms that the obtained result was not an artifact of the dimensionalityreduction procedure. Moreover, to ensure that such significance did not depend exclusively on the enhanced distance between the “rest” class and any other classes, we repeated the analysis with “rest” excluded from the accounted classes (p < 1e − 21, Supplementary Fig. 11c).
We performed a second test to compare the overall conservation of relative distances among the population vectors associated with different behaviors at the singlesession level. We captured the differences between the crossbehavioral distance matrices of two sessions with the Jeffries–Matusita metric and compared them with the bootstrap distribution obtained by shuffling the behavioral labels (see Methods for details). This was done for each pair of sessions within and across animals, both in the highdimensional recording space and in the LEM space, and by excluding the “rest” class from the test (Supplementary Fig. 11d, Fig. 3b, and Supplementary Fig. 11e, respectively). In all of these cases, the similarity between the polytopes of different sessions and animals was above chance for most session pairs, and nonsignificance often occurred for animals with a low SNR (cf. Supplementary Fig. 10b). Finally, we wondered whether the observed similarity between polytopes might have been induced by the sheer presence of somatotopy along the sensorimotor cortex. In fact, in the case of behaviors selectively engaging different parts of the sensorimotor cortex, somatotopy would generate a coarsegrained covariance structure between somatotopic regions that is similar across animals. However, the spontaneous behaviors considered here are complex and engage the full sensorimotor cortex (cf. Supplementary Fig. 6). This makes such a scenario unlikely. Nevertheless, to rule out this hypothesis, we divided the recorded units according to their somatotopic region (Supplementary Fig. 11e) and repeated the analysis from Supplementary Fig. 11d while shuffling the identities of the units recorded in the same somatotopic region across behaviors (i.e., the identities of the neurons recorded in the same somatotopic region were shuffled when building each of the average population vectors representing the six considered behaviors). This destroyed the covariance of neuronal activity across behaviors at the singleunit level, but maintained both the covariance of neuronal activity across behaviors at the somatotopicregion level, and the average firing rate of the different somatotopic regions. With the shuffled data, no session pair comparison reached an average pvalue below 0.05 (Supplementary Fig. 11g), demonstrating that the obtained results on polytope similarity across animals were not the byproduct of somatotopy.
Crosssubject and crosssession decoding
Polytopes capture the distance between the average neuronal representations of the different behaviors but neglect their shape and extension on the manifold. Encouraged by the similarities observed among the polytope structures of different recording sessions, we decided to perform a stronger test and attempted crosssubject decoding. Crosssubject decoding requires not only an agreement between the average representation of behaviors but also accounts for the variability in neuronal representations associated with each behavior. While it is impossible to find a direct correspondence at the singleneuron level across animals, similarities in lowerdimensional population structures can be used for crosssubject and crosssession decoding (Fig. 4a). For the decoding analysis, we divided the six behavioral classes (Fig. 4b) into two disjointed sets: one “alignment set”, which was used to align the neural structures, and one “decoding set”, which was used for training and testing a classifier. Thereby we ensured that no class was used for aligning the structures and classification in the neural space at the same time. The mean neural vectors (four dimensions) corresponding to the behavioral classes in the alignment set were used to compute a Procrustes transformation between two sessions to align the population activity structures^{27,28} (Fig. 4c). Procrustes transformations involve translation, scaling, reflection, and rotation and thus preserve the shape of a set of points. For decoding, we trained a classifier on samples from the decoding set of one session for a single rat using the activity in the dimensionalityreduced neural space as input. Then, we tested the generalization on another session of the same rat (crosssession decoding) or another rat (crosssubject decoding) (Fig. 4c). Notably, the samples of the decoding set of the two tested sessions were not used for computing the Procrustes transformation. In the first experiment, the alignment set consisted of four behavioral classes, with two other classes remaining for the decoding set. This resulted in a total of 15 possible splits into two sets. Classifiers trained on highly decodable sessions also successfully generalized to other sessions from the same or other rats (Fig. 5a–c and Supplementary Fig. 14a–b). In the generalization matrix (Fig. 5a), 13.88% of the generalization results (275 out of 45*44 = 1980) had a mean perclass accuracy higher than 60%, and 59 higher than 65%. In the set of sessions with the highest 10% signaltonoise ratio (SNR > 4.33), the mean perclass accuracy in the generalization task was μ = 59.7%, σ = 5.4. The bestperforming sessions included sessions of Rats A, B, and C with sufficient recording quality and a sufficiently high number of units for a robust estimation of the underlying population structures (Fig. 5d). Additionally, the correlation between withinsession and betweensession accuracies was high (Fig. 5c, n = 45, Pearson’s ρ = 0.68, p < 1e − 6). We defined the “generalization accuracy” of a session as the average test accuracy across all sessions (mean value per row of Fig. 5a first matrix). These generalization accuracies were correlated to the total number of units (Pearson’s ρ = 0.38, p < 0.01), with a higher number of units leading to a better estimation of the population structure. The generalization accuracies were also correlated to the session length (Pearson’s ρ = 0.37, p < 0.05) since the number of samples used for LEM (which included only time points with sufficient activity) varied across sessions and rats. Finally, the recording quality—namely, the SNR averaged over units—was correlated with generalization (Pearson’s ρ = 0.38, p < 0.01). Particularly, Rats A and B, which performed best in the generalization, had both a high SNR and a high total number of units (Fig. 5d).
Decoding is robust to methodological and classselection changes
To evaluate whether our results depended on a specific dimensionalityreduction method, we repeated the analysis using Isomap, another nonlinear dimensionalityreduction method, and principal component analysis (PCA). Also Isomap revealed neuronal structures that were comparable across subjects (Supplementary Figs. 12 and Fig. 5a second matrix). In contrast, linear methods such as PCA were not powerful enough to extract these neuronal structures, as they search for projections in the directions of largest variance (which can be biased by outliers) and are not optimized to maintain the local neighborhood structure of the highdimensional neural space (Fig. 5a third matrix); the LEM results were significantly better than those from data reduced using PCA (Wilcoxon signedrank test W = 167978303, n = 30375, p = 0 twosided). These results highlight that the activity of the sensorimotor cortex during spontaneous behavior evolves over a nonlinear manifold embedded in the original highdimensional neural space.
For a more systematic test of the relationship between the number of units and generalization, we took all sessions with a generalization accuracy of at least 55% (a total of 19 sessions from Rats A, B, and C) and conducted an ablation study with LEM reductions on the reduced number of units (20, 40, 60, and 80 units removed per session). We then repeated the generalization experiment on the aligned LEM structures. The accuracies steadily decreased with fewer units (Supplementary Fig. 13), confirming the high relevance of the number of units for a robust estimate of the population structure.
To determine which sensorimotor areas were most relevant for generalization, we again took the 19 best sessions and conducted LEM reductions after removing M1, S1, or M2. Additionally, for a fair comparison, we removed a random portion of all of the other units for underrepresented areas such that the number of units after the removal of M1, M2, and S1 neurons remained constant across sessions. Generalization accuracies on aligned LEM structures decreased considerably after the removal of M1 (accuracies μ = 53.9%, σ = 7.5%); these values were slightly but significantly lower than after the removal of units from S1 (accuracies μ = 55.9%, σ = 7.7%, Wilcoxon W = 5497776, n = 5415, p < 1e − 56), M2 (accuracies μ = 55.0%, σ = 8.3%, W = 6341443, p < 1e − 17), or of the same number of units distributed over all areas (accuracies μ = 55.6%, σ = 7.8%, W = 5659147, p < 1e − 47).
In a second experiment, we used only three classes in the alignment set and the three remaining in the decoding set to test the generalization under more difficult conditions, resulting in 20 possible splits of the six classes in total. The general pattern of the generalization matrix stayed the same (Supplementary Fig. 14c–d). To verify that the classifiers did not only learn to discriminate the simplest difference—the difference between rest and movement—we conducted another experiment without the class “rest”. Although the accuracies were lower in this setting, the general pattern remained the same (Supplementary Fig. 14e–f). To assess the relevance of the alignment of neural structures, we also tested the generalization on neural structures without explicit alignment as a control. In most cases, the accuracies on aligned structures were much higher than those on unaligned structures (Supplementary Fig. 15).
To further explore our results, we performed control classification experiments in which we examined the generalization on shuffled data (cf. Supplementary Fig. 5). Accuracies of shuffled data were significantly lower than those computed in the original LEM space (Wilcoxon W = 116677979, n = 30375, p = 0 when comparing with neuronshuffled, W = 115385202, p = 0 with timeshuffled, and W = 123943794, p = 0 with timeshifted data) and did not exceed the chance level (Supplementary Fig. 16). Furthermore, we computed LEM reductions in the nonbinarized neural space and repeated the generalization experiment. Also in this case we could find significant generalization for multiple sessions, but with accuracies lower than when the analyses were performed on binarized spikes (Supplementary Fig. 16d, W = 186661468, p < 1e − 180). This is probably because the binarization acts as a form of normalization that reduces the impact of overall firing rate differences between neurons, and could suggest that similarities are best captured by the differentiation between subgroups of units that are coactive during the different behaviors than by exact singleunit firing rate modulations.
Discussion
In this study, we investigated singleneuron activity as well as population activity patterns in the rats’ sensorimotor cortex during unconstrained and selfpaced behavior. The behavior was as closely related as possible to naturally occurring behavior, as it was based on foraging, but it was still performed in a limited arena to allow for reliable movement tracking. The first analyses were sanity checks to validate our approach of studying freely moving animals without a clear trial structure. Based on the chosen measure, STAPSSS, 54% of all neurons were significantly coupled to paw movements. This fraction of coupled neurons is in the range of previously reported numbers. For example, 60% of neurons in the hindlimb motor cortex reacted to different locomotion scenarios^{29}, and 44% in M1 were bodycoupled in freely moving rats^{25}. Our multiside recording approach allowed us to comprehensively test for differences in neuronal activity across the entire sensorimotor cortex. Previous research has found that the laterality of forelimb representations increases from M2 to M1 in a pedal task for headrestrained rats^{30}. Here, we extended this laterality gradient to more posterior regions—in particular, S1. As we targeted the output layer of the cortex (layer V), we putatively biased our recordings toward pyramidal tract neurons, which have been described as being predominately involved in laterality^{30}.
While the abovedescribed findings refer to the general features of the sensorimotor cortex, the main finding of our study was based on conserved neuronal population structures. Experimental, computational, and theoretical work has identified a rich structure within the coordinated activity of interconnected neural populations in movement control, decisionmaking, and memory tasks. These findings are conceptualized within the framework of neural population dynamics, which can reveal general motifs^{2}. Recurrent neural networks (RNNs) can be applied to neural data to reveal structural and geometric properties^{31}. Multiple tasks can then be represented in different RNN models. In these networks, some clusters of units have been identified as specialized for subsets of tasks^{1}. Alternatively, methods such as PCA and its variants dPCA and jPCA have been applied to identify the stability of motifs across modalities such as arm and speech control^{3}, as well as within and across brain areas^{4}. In contrast to the previously described studies, we focused on the existence of conserved neuronal structures across animals without any clear instructed task line but with several behavioral classes. These two points differentiate our study from previous publications in the field. We investigated population activity patterns, which are commonly assumed to reside on lowdimensional manifolds in the full neural state space^{14,16,32,33,34}. In contrast to the (globally) linear method like PCA that most studies have used^{4,12,15,16,17,18,20,35}, we assumed the preservation of local neighborhood relations in the data. Therefore, we employed LEM^{5,22} to reveal the presumed preserved lowdimensional structures. Remarkably, neuronal population activity during unconstrained behavior contained similar structures across animals and sessions, already visible in the first three dimensions. Furthermore, the distribution of different behaviors across lowdimensional neural structures was systematic, which we confirmed with our above chance, withinsubject decoding results. The allocation of different behaviors on the population structures revealed strong similarities across rats. Particularly, movement and rest could be clearly visually distinguished in the first dimension. This is in line with results on clear separations in the neural state space for outputpotent and outputnull (e.g., preparatory) neural activity^{12,20}.
To support our main claim that lowdimensional neural manifolds are comparable across sessions and animals even in the case of unconstrained behavior, we first showed with our polytope analysis that the relative positions of the neural representations associated with different behavioral classes were conserved across animals and sessions above chance. The analyses of the polytope structures compared the distance between the average neuronal representations of behaviors, neglecting their precise spatial extension on the manifold as determined by the variability in neuronal representation of each behavior. Conversely, crosssubject decoding was also affected by such variability and, therefore, tested an even stronger degree of similarity. Ultimately, we evaluated the performance of a classifier trained on the neuronal activity of one subject to predict the behavior of another based on its own neuronal activity; this provided us with a proxy to experimentally test and quantify the degree of universality of neural representations across subjects. Since the neuronal state space of different subjects cannot be directly compared (given the difference in number and identity of the recorded neurons), applying dimensionality reduction and alignment was necessary to achieve this goal. A simple, supervised, shapepreserving alignment procedure—namely, a Procrustes transformation between mean population vectors for different behavioral classes in the dimensionalityreduced neural space—sufficed for successful crosssubject generalization in a decoding task with distinct but related behavioral classes. Our procedure was applicable to sessions with sufficient recording quality (indicated by a high SNR of the recorded units) and enough units for robust population estimation. Further, the generalization accuracies of the sessions were closely related to their withinsession accuracies. Generalization was considerably worse for population structure estimates based on fewer units. In line with the withinsession decoding results, we also found that generalization significantly decreased after the removal of M1, which indicated consistent population responses especially in this area. The low relevance of the anterior motor cortex to information regarding behavioral categories is in line with our STAPSSS results. Nevertheless, in contrast to the encoding of paw movements, our results on the population decoding of the higherlevel behavioral categories hinted at major contributions from M1, not only S1. Thus, our results close a gap in a previous study that investigated postural and behavioral encoding in the posterior parietal cortex and M2^{36}. While we mostly used LEM as a dimensionalityreduction method given its solid theoretical basis, we also showed that another nonlinear dimensionalityreduction method, Isomap, can be used to reveal neural structures that are comparable across subjects.
A shared structure in neuronal activity across subjects has been shown mostly in fMRI studies, where even betweensubject classification has been demonstrated^{27,37,38,39,40,41}. While these works have focused on watching movies (an activity that can be conducted similarly for different subjects), EEG and EMG crosssubject decoding has been shown for hand movements^{42,43}. In rodents, related work has shown the crosssubject decoding of odor sequences in the orbitofrontal cortex^{44} and placecell activity in the hippocampus^{45}. Here, the presentation of external stimuli and the presence of a precise trial structure might facilitate the detection of an operational manifold common to all animals. In contrast to these studies, we showed crosssubject classification in a more complex case where rats roamed freely without training or a trial structure in the underlying task. Therefore, our main finding of shared neural structures is consistent with recent findings but also extends them to more complex, less constrained behavior. To our knowledge, this is the first time that the conservation of neural structures across animals and for distinct, spontaneous behavioral classes has been shown. This finding implies that conserved neuronal structures occur without training. Therefore, the neuronal computations underlying these structures might be similarly realized across individuals, either from birth or during development.
The similarity of neural population structures in the sensorimotor cortex is of great relevance for the development of BCI systems, which are built to aid physically disabled people and, thus, often target the sensorimotor cortex^{46}. Essentially, BCI systems aim to map neuronal activity to targeted movements by first transforming the noisy and nonstable recorded singleunit neuronal activity (which strongly varies across a longer recording time span) to a, supposedly, more stable lowerdimensional space, and then using a linear or nonlinear decoder mapping to movement classes or trajectories. Gallego et al. showed that in monkey sensorimotor cortex, it is possible to find an alignment between lowdimensional manifolds during cursor tasks, even when they are reconstructed from recordings interspaced by long periods of time^{35}. Following these lines, the topic of the alignment of lowdimensional neural manifolds across time for the longterm stability of BCI systems has recently gained focus, fostering the development of new analytical techniques^{47,48,49,50}. Once the reconstructed manifolds are aligned to a common frame, the decoder mapping between neuronal activity and movement classes can be fixed in time, which diminishes patients’ discomfort caused by the constant retraining of the BCI system. While progress has been made in the alignment of manifolds across time for a single subject, the crosssubject generalization of neural motor patterns has not been shown so far. If the neuronal manifolds of the sensorimotor cortex were subjectspecific, BCI systems would need to learn the mapping from the neuronal manifold to the movement space from scratch for each subject. For deep artificial neural networks, which, today, are often used as the most powerful nonlinear mapping methods, the training phase is computationally expensive, often requiring hours of GPU training^{51}; in contrast, once a network is trained, it can be used in the prediction phase with low computational effort. In this manuscript, we demonstrate that the neuronal manifolds of the sensorimotor cortex associated with spontaneous behavior are not only stable in time but are also conserved across animals. This implies that BCI training time can be substantially reduced by preusing data from other subjects, in which case only a short finetuning phase would be necessary. Our work provides a proofofprinciple for the crosssubject generalization of complex selfpaced behaviors that involve the whole body, with recordings of a few singleunits (in the range of 100–200 neurons per session) from only partially overlapping areas of the sensorimotor cortex provided as data. While it has been hypothesized that crossindividual decoding might not be possible with increasing task complexity^{5}, our results indicate that even during unconstrained behavior, the relationships among neural activity patterns are conserved across different animals. This conservation of populationlevel neural phenomena provides a foundation for crosssubject decoding, even in the difficult case of unconstrained behavior.
Methods
Animal surgery
We implanted six male Long Evans rats at the age of eight weeks with 22 tungsten electrodes (200 to 600 kOhm impedance, polyimide insulation, WHS Sondermetalle, Grünsfeld, Germany) at a 1.2 mm implantation depth in each hemisphere (implantation: January 2017 for Rat F, April 2017 for Rats A–E). Recordings were taken in JuneAugust 2017 (see Supplementary Table 1). The relatively long implantation time (perfusion in November 2017), as well as the amount of electrode wires, led to tissue growth around the electrodes and changes in the original implantation depth. We, thus, cannot provide histological pictures. Electrode locations spanned from –2 to +5 mm in the anterior/posterior direction and from 1 to 4 mm in the lateral/medial direction. This resulted in three medial–lateral rows of six electrodes each, plus one row of four electrodes (see Fig. 1b). More information regarding the procedure can be found in ref. 52. The Regierungspräsidium Freiburg, Abteilung Landwirtschaft, Ländlicher Raum, Veterinär und Lebensmittelwesen approved all animal procedures.
Behavioral task
The rats were kept waterrestricted for the time course of the experiments (free access to water for two days per week). For the experiments, the rats moved unconstrained on a mesh of 30 × 40 cm in a closed arena. Every 10 to 30 s, a waterspout pseudorandomly positioned by two servo motors released a drop of water onto the mesh, which the animals could find and consume. To prevent the rats from merely following the movements of the waterspout, we included dummy movements that were not followed by a release of water. Even experienced animals were not able to predict the position of water drops without an active search, and the animals did not find all water drops throughout a session. This task has been previously described^{52}. Here, we only used part of the dataset discussed in ref. 52; in particular, we only included sessions with a minimum duration of 30 min.
Data acquisition and the preprocessing of extracellular recordings
Extracellular signals were recorded at 30 kHz and bandpass filtered, amplified, and digitized using a head stage (Intan Technologies, Los Angeles, California) situated at the head of the animal. Spike sorting was conducted on highpass filtered signals (cutoff at 300 Hz) separately for each electrode. Spikes were defined as amplitude threshold crossings of four times the standard deviation of the signals. For each spike, we extracted the window of −0.5 to 2 ms around the peak amplitude (resulting in 76 values per spike). Spike sorting consisted of two phases for each unit. First, a seed spike was estimated. This was accomplished by calculating the spike neighborhoods (spikes within the average noise level, half a millisecond before the spike, across all units) for 500 randomly chosen spikes. The spike with the most neighbors was chosen as the seed spike. Second, we optimized the spike waveform through an iterative procedure. This was done by alternating the calculation of a new noise level for the neighboring spikes, the update of the neighborhood (spikes within the new noise level), and the update of the average waveform. This iterative procedure ended when the neighborhood assignments remained constant. The algorithm proceeded with the remaining spikes by choosing a new seed spike. For our singleunit analysis, we only kept singleunits according to the distribution of interspike intervals. singleunits with a firing rate lower than 0.1 Hz were not included in the analysis. Two cameras (Stingray, F033C IRF CSM, Allied Vision Technologies) positioned below the mesh tracked the movements of the colored paws. The videos were taken with a frame rate of 80 Hz and smoothed with a Gaussian filter before analysis.
Singleunit STAPSSS analysis
Paw movements were labeled as “swing” for horizontal velocities higher than 0.3 mm per 10 ms (the bin size we used for our analysis) and “stance” otherwise. Spikes were also binned with a bin size of 10 ms. For each neuron and each paw, we defined the spiketriggered average paw swing–stance status (STAPSSS) as the behavioral average over all windows ±1 s around the spikes. We normalized each STAPSSS waveform by the mean. We defined the paw coupling of a neuron as the ratio of the standard deviation of the STAPSSS waveform to the statistical control standard deviation. The latter was defined as the 0.99 quantile standard deviation of a distribution constructed out of the standard deviations of the STAPSSS waveforms of 1000 randomly shifted spike trains. If a neuron was not related to a paw’s movement, its STAPSSS waveform would be flat and its standard deviation would not exceed the control standard deviation. We defined the contralateral bias as the ratio of contralateral to ipsilateral paw coupling. Statistical analyses were done using the anovan, multcompare, and ttest Matlab functions. The ANOVA tests always included the rat’s ID as an additional factor. The analyses of this manuscript were performed with Matlab 2019a, Python 3.6, pandas 0.24.1, matplotlib 3.0.0, numpy 1.14.5, scikitlearn 0.20.0, scipy 1.1.0, and tensorflow 1.10.0.
Decoding from spike trains
We used fully connected neural networks with three hidden layers of 500 units each for decoding. The networks’ inputs were the Gaussiansmoothed (σ = 20 ms) binned spikes in ±400 ms, resulting in 81 input bins for each neuron. In contrast to the STAPSSS analysis, where only singleunits were considered, we used all units as input for decoding. Each session was split into training, validation, and test sets (70/15/15%). Two of the 106 sessions were excluded from decoding because of insufficient data. Training was conducted with the Adam optimizer^{53}, batch size 64, and an initial learning rate of 0.0001. A dropout rate of 75%, L2 regularization (λ = 1e − 4), and early stopping were applied to prevent overfitting. To deal with class imbalance, we used weighted crossentropy loss to put more weight on the less frequent class (swing). The reported accuracies were mean perclass accuracies. The decoding accuracies of the deep neural network were significantly better than a baseline linear classifier (twosided paired ttest, t = 6.55, p < 1e − 8). For the baseline, we used a logistic regression with threefold crossvalidation of the L2 regularization strength on the concatenated training and validation sets. The test sets for each session were the same as for the artificial neural network. Class weights were adjusted to be inversely proportional to class frequencies, as for the artificial neural network. The artificial neural network was implemented in Tensorflow. For the linear baseline, we used Python’s scikitlearn function LogisticRegressionCV.
Dimensionality reduction
We used LEM^{5,22}, an unsupervised nonlinear dimensionalityreduction method, to investigate the lowdimensional structure of population activity. For each session, spike counts were binned in 100 ms bins and then binarized (1 for at least one spike per bin, 0 for no spikes). Single and multi units were used. Only time points with at least 15 active units were retained. Since we restricted further analysis to sessions with at least 5000 valid time points, we considered only 95 of the 106 sessions. For each session, we constructed an unweighted, mutual kNN graph based on the Hamming distance on the columns of the n × t matrix (n units, t time points). Our code for LEM was built on recent work^{5}. Two iterations of the LEM algorithm were performed. However, in contrast to Rubin et al., we used the Hamming distance in the first iteration and reduced to 20 dimensions. In the first iteration, we used 0.5% of the time points as neighbors; in the second, this parameter was set to 7.5%. Furthermore, we applied a random walk normalized graph Laplacian instead of the symmetric normalized graph Laplacian, as proposed in a previous study^{54}. In detail, we constructed the unnormalized graph Laplacian as L = D − W, with D as the diagonal degree matrix and W as the adjacency matrix of the kNN graph. Solving the generalized eigenvalue problem Lv = λDv corresponded to finding the first eigenvectors of the random walk normalized graph Laplacian L_{n} = D^{−1}L^{54}. Since the eigenvector corresponding to the smallest eigenvalue (zero) is constant, we discarded the first dimension of the LEM for all analyses and decoding studies. The other LEM eigenvectors (i.e., dimensions) were ordered by eigenvalue magnitude—that is, the “splitability” of the time points in different clusters (i.e., the dimensions that best divided the time points into clusters came first). For the LEM reductions on units from different sensorimotor areas, we randomly chose 20 units from each area as input (if fewer than 20 units for an area were available, the analysis was omitted). We chose to reduce to six dimensions in the LEM space, leaving us with five dimensions for decoding with deep neural networks (as mentioned above, the first dimension of the LEM must be discarded). For the ablation study on sessions with 20, 40, 60, or 80 units removed, we reduced to 20 dimensions in the first two and 10 dimensions in the second two cases (in these latter cases, we did not have enough neurons left to retain high dimensionality in the LEM space). For the study on LEM reductions after the removal of sensorimotor areas, we removed \({n}_{{{{{{\mathrm{max}}}}}}}=\max\)(#M1, #M2, #S1 units) from each area for each session. For underrepresented areas, we additionally discarded n_{max} − n_{area} randomly chosen units. As before, given the lower number of neurons, we reduced to 10 dimensions. To investigate the dimensionality of the LEM space using the method of ref. 5, we computed the average number of neighbors of all time points in the 20dimensional LEM space in circles with increasing radii. The dimensionalities were then obtained as the slope of a line around the steepest point in a log–log plot of neighbors against radii.
For the dimensionality reduction with Isomap, we used Landmark–Isomap^{55}, which is more efficient for very large datasets. We set the number of neighbors to 0.5%, as for the LEM, and used 10% of the time points as landmarks. PCA reductions where computed on nonbinarized spikes.
Behavioral labeling
We used the freely available tool MuViLab for the behavioral labeling of the videos. Two human annotators who were blinded to the neural data manually labeled the 48 sessions divided into 500 ms snippets. The 48 sessions were chosen based on them having a clear saddlelike shape and at least five significantly coupled units: Rat A—13 sessions recorded between 2017/06/08 and 2017/08/03, Rat B—16 sessions recorded between 2017/06/01 and 2017/08/21, Rat C—seven sessions between 2017/06/01 and 2017/06/29, Rat D—six sessions between 2017/06/08 and 2017/07/11, Rat E—three sessions between 2017/06/08 and 2017/06/22, and Rat F—three sessions between 2017/06/07 and 2017/06/30. The criteria for the behavioral classes were as follows: Step—the rat moved at least one paw but did not drink or rear at the same time; turn—the rat moved its head; drink—the rat drank from the spout or collected water drops from the mesh with its mouth; groom—the rat performed typical grooming movements; rear—the rat stood on its hind paws; rest—the rat showed no obvious movements. In rare cases, samples were excluded from labeling when the behavior of the rat was not visible because it was located near the borders of the arena. Examples of the different behaviors can be found at https://gin.gnode.org/optophysiology/Conserved_structures_cortex.
Singleunit behavioral coding
To establish the singleunit coding of a specific behavior or stimulus, it is common practice to compare the average firing rate of the unit prior to the event (baseline) and after it (response). In the case of selfinitiated behaviors, however, it is difficult to unambiguously identify temporal windows that can be associated with a baseline or response. Thus, we tested whether a unit increased its firing rate during each of the six behavioral categories and compared this rate to the unit’s firing during the remainder of the recorded time. The test was performed using a Wilcoxon ranksum test with Benjamini–Hochberg correction for multiple comparisons and α = 0.05.
In a second analysis, we aimed to compare the diversity in singleunit firing rates during two behaviors with the distance in the LEM space of the population vectors associated with such behaviors. To obtain the number of singleunits that changed their firing rates during different behaviors, we divided the spike counts (500 ms binning) of each unit according to the six behavioral classes and performed a Kruskal–Wallis test. When the main effect was significant, we performed a posthoc analysis to selectively compare the unit firing rates during each pair of behaviors. Significance was fixed at 0.05. Since the final aim of this analysis was to compare the average number of units that changed rates with the distance in the LEM space of the population vectors associated with different behaviors, we did not want the unequal sample size of the behavioral classes to affect the significance of the posthoc tests. Therefore, before performing the Kruskal–Wallis test, we randomly selected an equal number of samples (equal to the sample size of the smallest class) from all behavioral classes for each unit. We then repeated the test 100 times and computed the average number (first across the 100 samplings and then across the session’s units) of significant posthoc tests obtained for each class comparison and each session. Supplementary Fig. 9a displays their average across sessions.
Similarities among behavioral representations across sessions
To investigate whether the relative positions of the neural representations associated with different behavioral classes were conserved across sessions and animals, we computed the Euclidean distance between the average behavioral population vectors of a session, then tested whether these distances were more similar to those observed in other sessions than to what would be expected by randomly shuffling the behavioral state labels. This was performed by first comparing the ranked distances between the polytope vertexes and then comparing the actual distance values. For each session v, we computed the Euclidean distance \({D}_{ij}^{v}\) between the average population vector p^{i} and p^{j} of all pairs of behavioral classes i and j. Then, for each behavioral class i, we ranked the remaining classes j according to their distance \({D}_{ij}^{v}\) from i. For each pair of sessions v and w and each class i, we accounted for the similarities in ranked distances by defining the statistic \({s}_{i}^{vw}\) as the number of classes matching the same rank in the two sessions. For the six behavioral classes, \({s}_{i}^{vw}\) ranged between a maximum value of 5 (perfect match) to a minimum value of 0 (no match). The distribution of \({s}_{i}^{vw}\) across all sessions was compared with a bootstrap distribution in which the same statistic, s_{boot}, was computed over two random permutations of the numbers from 1 to 5. With six possible classes, there are 5! = 120 possible permutations of the remaining five classes, giving \(\left(\begin{array}{c}5!\\ 2\end{array}\right)+5!=7260\) unordered pairs of random permutations. We thus used the Kolmogorov–Smirnov test to compare the distribution between the observed \({s}_{i}^{vw}\) (n = 990 session pairs) and bootstrapped s_{boot} (n = 7200) similarities.
The analysis described in the previous paragraph tested whether the distances between the pairs of behaviors (polytope vertexes) had a similar order (e.g., from the closest to the furthest) for different sessions or animals. To compare the actual distance values, we computed the matrix of pairwise Euclidean distances D^{v} between the average class population vectors p^{i} in the LEM space. Then, for each other session w, we performed a Procrustes transformation to rescale the behavioral population vectors of w with those of v and computed the distance matrix D^{w} on the rescaled vectors. The Procrustes transformation did not affect the relative distance between vertexes but prevented differences in scale between the polytopes of different sessions from obscuring the quantity of interest. To quantify whether the set of relative distances between behavioral classes was, to some extent, maintained across sessions, we computed the difference between D^{v} and D^{w} as the Jeffries–Matusita distance \({d}_{JM}({{{{{{{{\bf{D}}}}}}}}}^{v},{{{{{{{{\bf{D}}}}}}}}}^{w})=\sqrt{{\sum }_{i,j}{(\sqrt{{D}_{i,j}^{v}}\sqrt{{D}_{i,j}^{w}})}^{2}}\), where i and j were indexes running over the six classes, and compared this difference with what we would obtain by chance. We employed the Jeffries–Matusita metric because it reduces the effect of outliers, but similar results were found with a Euclidean metric as well. The distribution d_{JM}(D^{v}, D^{w}) obtained with the original distance matrices was tested against the bootstrap distribution \({{d}_{JM}({{{{{{{{\bf{D}}}}}}}}}^{v},{{{{{{{{\bf{D}}}}}}}}}_{b}^{w})}_{1...n{{{{{\mathrm{bootstraps}}}}}}}\) obtained by randomly permuting the behavioral labels associated with the population vectors of the session w. For each session pair (v, w), we then compared d_{JM}(D^{v}, D^{w}) with those obtained on the bootstrapped \({{{{{{{{\bf{D}}}}}}}}}_{boot}^{w}\) and computed a pvalue for the H_{0} of d_{JM}(D^{v}, D^{w}) that was obtained by chance. The bootstrap sample included all possible class label permutations (n = 720). Figure 3b and Supplementary Fig. 11d show the significance of the comparison of each session pair when computed on all six behavioral classes, and Supplementary Fig. 11e shows the same but with the “rest” class excluded. Finally, to ensure that the observed similarity between the polytopes of different animals was not the byproduct of somatotopy but reflected the generalization of the behaviorspecific covariance structure at a finer scale, we repeated the analysis described in the previous paragraph, shuffling the identities of the units recorded within the same somatotopic region. In particular, for each session, we first divided the recorded units according to their somatotopic region (adapted from refs. 56,57) as indicated in Supplementary Fig. 11f. Then, to compute D^{v}, we composed the average population vectors p^{i} associated with each behavior, this time by randomly shuffling the identities of the units recorded in the same somatotopic region separately for each behavior. For example, after shuffling, if the third entry of the vector p^{step} corresponded to a unit x recorded in the motor region for the control of forelimbs, the third entry of p^{rear} was a unit recorded in the same region but possibly with different identity y. In this way, we removed the relation between the fine coding information of each behavior within somatotopic regions, while maintaining the regionspecific average firing rate and the relation of the average region information between behaviors. We repeated the analysis for 500 random permutations of unit dispositions on the p^{i} vectors. As in Supplementary Fig. 11d, for each permutation, we obtained a pvalue following the same bootstrap procedure described above. Supplementary Fig. 11g shows pair comparisons with average pvalues below (white) or above (black) 0.05.
Populationlevel decoding
We trained one deep neural network per session to classify the six behavioral classes given the 10dimensional neural data in seven bins with 100 ms each as input. The data was min–max normalized (min and max were only calculated on training sets). The deep network architecture and training were almost identical to the network used for the decoding task above. However, we used only 200 units per layer and a dropout rate of 25%, and we chose a crossvalidation strategy to deal with unbalanced classes. In the latter step, the available data was split into four parts of equal size. Four runs were conducted per session, using two parts as the training set, one as the validation set for early stopping, and the fourth as a test set. The final test results were calculated as the mean over all four test sets and runs. As for the decoding of the swing–stance status, we used weighted crossentropy loss (more weight on less frequent classes) to deal with the class imbalance. All accuracies that we report were meanperclass accuracies (balanced accuracies) to ensure that more frequent classes did not bias the results. While we used 10 dimensions for this behavioral decoding task—in line with the estimated dimensionality—only five dimensions remained for the areaspecific dimensionalityreduced data since the lower number of neurons did not allow for a reduction in a higherdimensional LEM space. For the supervised alignment procedure, we always restricted the analysis to four neural dimensions to avoid underdetermination (that is, the remaining dimensions provided by LEM were not used—no completely new dimensionality reduction was computed). We used Matlab’s Procrustes function to find a transformation between class means. Proper transformation was important because of the sign ambiguity of eigenvectors, which might otherwise have led to different orientations of the neural structures. Before alignment, both neural structures were normalized to the 0–1 range. An SVM with a Gaussian kernel (Matlab fitcecoc) was used as classifier. Training was conducted with an equalized number of samples per class (i.e., the class with the fewest samples determined the number of samples taken from each class) and default parameters (kernel size 1). For the SVM classification, we did not use fourfold crossvalidation as we did for the classification of neural networks (see above). Instead, we performed 20 repetitions with different samplings of the training set (Monte Carlo crossvalidation).
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.
Data availability
One example dataset used in this study has been deposited in the GitHub database at [https://doi.org/10.5281/zenodo.7296960]. The full data are available from the corresponding author upon reasonable request. Video examples of the rats showing different behaviors can be found at [https://gin.gnode.org/optophysiology/Conserved_structures_cortex]. The data used to generated the figures of this study are provided in the Source Data file. Source data are provided with this paper.
Code availability
Code for the most important functions is available at https://doi.org/10.5281/zenodo.7296960^{58}.
References
Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T. & Wang, X.J. Task representations in neural networks trained to perform many cognitive tasks. Nat. Neurosci. 22, 297–306 (2019).
Vyas, S., Golub, M. D., Sussillo, D. & Shenoy, K. V. Computation through neural population dynamics. Ann. Rev. Neurosci. 43, 249–275 (2020).
Stavisky, S. D. et al. Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis. Elife 8, e46015 (2019).
Perich, M. G., Gallego, J. A. & Miller, L. E. A neural population mechanism for rapid learning. Neuron 100, 964–976 (2018).
Rubin, A. et al. Revealing neural correlates of behavior without behavioral measurements. Nat. Commun. 10, 1–14 (2019).
Rabinovich, M., Huerta, R. & Laurent, G. Transient dynamics for neural processing. Science 321, 48–50 (2008).
Shenoy, K. V., Sahani, M. & Churchland, M. M. Cortical control of arm movements: a dynamical systems perspective. Ann. Rev. Neurosci. 36, 337–359 (2013).
Michaels, J. A., Dann, B. & Scherberger, H. Neural population dynamics during reaching are better explained by a dynamical system than representational tuning. PLoS Comput. Biol. 12, e1005175 (2016).
Buonomano, D. V. & Maass, W. Statedependent computations: spatiotemporal processing in cortical networks. Nat. Rev. Neurosci. 10, 113–125 (2009).
Fetz, E. E. Are movement parameters recognizably coded in the activity of single neurons? Behav. Brain Sci. 15, 679–690 (1992).
Churchland, M. M. et al. Neural population dynamics during reaching. Nature 487, 51 (2012).
Kaufman, M. T., Churchland, M. M., Ryu, S. I. & Shenoy, K. V. Cortical activity in the null space: permitting preparation without movement. Nat. neurosci. 17, 440 (2014).
Sadtler, P. T. et al. Neural constraints on learning. Nature 512, 423 (2014).
Gallego, J. A., Perich, M. G., Miller, L. E. & Solla, S. A. Neural manifolds for the control of movement. Neuron 94, 978–984 (2017).
Miri, A. et al. Behaviorally selective engagement of shortlatency effector pathways by motor cortex. Neuron 95, 683–696 (2017).
Gallego, J. A. et al. Cortical population activity within a preserved neural manifold underlies multiple motor behaviors. Nat. Commun. 9, 4233 (2018).
Hennig, J. A. et al. Constraints on neural redundancy. Elife 7, e36774 (2018).
Russo, A. A. et al. Motor cortex embeds musclelike commands in an untangled population response. Neuron 97, 953–966 (2018).
Pandarinath, C. et al. Latent factors and dynamics in motor cortex and their application to brain–machine interfaces. J. Neurosci. 38, 9390–9401 (2018).
Elsayed, G. F., Lara, A. H., Kaufman, M. T., Churchland, M. M. & Cunningham, J. P. Reorganization between preparatory and movement population responses in motor cortex. Nat. Commun. 7, 13239 (2016).
Pandarinath, C. et al. Inferring singletrial neural population dynamics using sequential autoencoders. Nat. Methods 15, 805–815 (2018).
Belkin, M. & Niyogi, P. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003).
Musall, S., Kaufman, M. T., Juavinett, A. L., Gluf, S. & Churchland, A. K. Singletrial neural dynamics are dominated by richly varied movements. Nat. Neurosci. 22, 1677–1686 (2019).
Paxinos, G. & Watson, C. The Rat Brain in Stereotaxic Coordinates: Hard Cover Edition (Elsevier, 2006).
Kells, P. A., Gautam, S. H., Fakhraei, L., Li, J. & Shew, W. L. Strong neurontobody coupling implies weak neurontoneuron coupling in motor cortex. Nat. Commun. 10, 1575 (2019).
Braitenberg, V. & Schüz, A. Cortex: Statistics and Geometry of Neuronal Connectivity (Springer Science & Business Media, 2013).
Haxby, J. V. et al. A common, highdimensional model of the representational space in human ventral temporal cortex. Neuron 72, 404–416 (2011).
Chen, H.T., Manning, J. R. & van der Meer, M. A. Betweensubject prediction reveals a shared representational geometry in the rodent hippocampus. Curr Biol. 31, 4293–4304.e5 (2020).
DiGiovanna, J. et al. Engagement of the rat hindlimb motor cortex across natural locomotor behaviors. J. Neurosci. 36, 10440–10455 (2016).
Soma, S. et al. Distinct laterality in forelimbmovement representations of rat primary and secondary motor cortical neurons with intratelencephalic and pyramidal tract projections. J. Neurosci. 37, 10904–10916 (2017).
Russo, A. A. et al. Neural trajectories in the supplementary motor area and motor cortex exhibit distinct geometries, compatible with different classes of computation. Neuron 107, 745–758 (2020).
MacDowell, C. J. & Buschman, T. J. Lowdimensional spatiotemporal dynamics underlie cortex wide neural activity. Curr. Biol. 30, 2665–2680.e8 (2020).
Gao, P. & Ganguli, S. On simplicity and complexity in the brave new world of largescale neuroscience. Curr. Opin. Neurobiol. 32, 148–155 (2015).
Cunningham, J. P. & Byron, M. Y. Dimensionality reduction for largescale neural recordings. Nat. Neurosci. 17, 1500 (2014).
Gallego, J. A., Perich, M. G., Chowdhury, R. H., Solla, S. A. & Miller, L. E. Longterm stability of cortical population dynamics underlying consistent behavior. Nat. Neurosci. 23, 260–270 (2020).
Mimica, B., Dunn, B. A., Tombaz, T., Bojja, V. S. & Whitlock, J. R. Efficient cortical coding of 3d posture in freely behaving rats. Science 362, 584–589 (2018).
Kriegeskorte, N., Mur, M. & Bandettini, P. A. Representational similarity analysisconnecting the branches of systems neuroscience. Front. Syst. Neurosci. 2, 4 (2008).
Haxby, J. V., Connolly, A. C. & Guntupalli, J. S. Decoding neural representational spaces using multivariate pattern analysis. Ann. Rev. Neurosci. 37, 435–456 (2014).
Hasson, U., Nir, Y., Levy, I., Fuhrmann, G. & Malach, R. Intersubject synchronization of cortical activity during natural vision. Science 303, 1634–1640 (2004).
Guntupalli, J. S. et al. A model of representational spaces in human cortex. Cereb. Cortex 26, 2919–2934 (2016).
Chen, J. et al. Shared memories reveal shared structure in neural activity across individuals. Nat. Neurosci. 20, 115–125 (2017).
Zhang, G., Davoodnia, V., SepasMoghaddam, A., Zhang, Y. & Etemad, A. Classification of hand movements from eeg using a deep attentionbased lstm network. IEEE Sens. J. 20, 3113–3122 (2019).
Rabin, N., Kahlon, M., Malayev, S. & Ratnovsky, A. Classification of human hand movements based on emg signals using nonlinear dimensionality reduction and data fusion techniques. Expert Syst. Appl. 149, 113281 (2020).
Zhou, J. et al. Evolving schema representations in orbitofrontal ensembles during learning. Nature 590, 606–611 (2021).
Chen, H.T., Manning, J. R. & van der Meer, M. A. Betweensubject prediction reveals a shared representational geometry in the rodent hippocampus. Curr. Biol. 31, 4293–4304.e5 (2021).
Brandman, D. M., Cash, S. S. & Hochberg, L. R. Human intracortical recording and neural decoding for brain–computer interfaces. IEEE Transa. Neural Syst. Rehabilit. Eng 25, 1687–1696 (2017).
Degenhart, A. D. et al. Stabilization of a brain–computer interface via the alignment of lowdimensional spaces of neural activity. Nat. Biomed. Eng. 4, 672–685 (2020).
Dabagia, M., Kording, K. P. & Dyer, E. L. Comparing highdimensional neural recordings by aligning their lowdimensional latent representations. https://doi.org/10.48550/arXiv.2205.08413 (2022).
Karpowicz, B. M. et al. Stabilizing braincomputer interfaces through alignment of latent dynamics. Preprint at bioRxiv https://doi.org/10.1101/2022.04.06.487388 (2022).
Jude, J., Perich, M. G., Miller, L. E. & Hennig, M. H. Robust alignment of crosssession recordings of neural population activity by behaviour via unsupervised domain adaptation. Int. Conf. Mach. Learn. https://doi.org/10.48550/arXiv.2202.06159 (2022).
Thompson, N. C., Greenewald, K., Lee, K. & Manso, G. F. The computational limits of deep learning. Preprint at https://arxiv.org/abs/2007.05558 (2020).
Eriksson, D., Heiland, M., Schneider, A. & Diester, I. Distinct dynamics of neuronal activity during concurrent motor planning and execution. Nat. Commun. 12, 1–18 (2021).
Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. Preprint at https://arxiv.org/abs/1412.6980 (2014).
Von Luxburg, U. A tutorial on spectral clustering. Stat. Comput. 17, 395–416 (2007).
De Silva, V. & Tenenbaum, J. B. Global versus local methods in nonlinear dimensionality reduction. NIPS 15, 705–712 (2002).
Hall, R. D. & Lindholm, E. P. Organization of motor and somatosensory neocortex in the albino rat. Brain Res. 66, 23–38 (1974).
Neafsey, E. J. & Sievert, C. A second forelimb motor area exists in rat frontal cortex. Brain Res. 232, 151–156 (1982).
Melbaum, S. et al. Conserved structures of neural activity in sensorimotor cortex of freely moving rats allow crosssubject decoding. GitHub: Conserved_structures_cortex, https://doi.org/10.5281/zenodo.7296960 (2022).
Asiminas, A. Scidraw Scientific Drawings. https://doi.org/10.5281/zenodo.3926077, https://doi.org/10.5281/zenodo.3926015, https://doi.org/10.5281/zenodo.3926125, https://doi.org/10.5281/zenodo.3926277, https://doi.org/10.5281/zenodo.3926223, https://doi.org/10.5281/zenodo.3926237, https://creativecommons.org/licenses/by/4.0/ (2022).
Acknowledgements
We thank Krishna Shenoy, Philipp Schröppel, and Christian Zimmermann for their comments on the manuscript. We thank Antonis Asiminas and SciDraw for the illustrations of behaving rats (https://www.scidraw.io/category/rat)^{59}. This work has been funded as part of BrainLinksBrainTools which is funded by the Federal Ministry of Economics, Science and Arts of BadenWürttemberg within the sustainability programm for projects of the excellence initiative II, to I.D. and T.B. as well as the Bernstein Award 2012 and the Research Unit 5159 Resolving prefrontal flexibility, DI 1908/111 to I.D. DFG grant Du 354/101 supported D.D. The Boehringer Ingelheim Foundation grant ‘Complex Systems’ supported E.R.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Contributions
T.B. and I.D. designed the study. S.M. analyzed the data, except the parts that E.R. analyzed. E.R. performed the polytope and behavioral singleunit coding analyses. D.D. contributed to the design of the analyses and the statistical evaluation. S.M., T.B., I.D., E.R., D.D., and A.S. wrote and edited the manuscript. D.E. conceived and recorded the dataset.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Communications thanks the anonymous reviewer(s) for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Source data
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Melbaum, S., Russo, E., Eriksson, D. et al. Conserved structures of neural activity in sensorimotor cortex of freely moving rats allow crosssubject decoding. Nat Commun 13, 7420 (2022). https://doi.org/10.1038/s41467022351156
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467022351156
This article is cited by

Reconstructing computational system dynamics from neural data with recurrent neural networks
Nature Reviews Neuroscience (2023)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.