The behavior of a mouse in its environment can be considered as consisting of a set of modular building blocks, and these may coincide with modules of neural activity. “We had this ethological hypothesis . . . that the brain takes advantage of motifs of action in order to construct complex behavioral sequences,” says Sandeep Robert Datta from Harvard Medical School. He and his team, including first author Jeffrey Markowitz, have extended their earlier work on MoSeq to search for neural correlates of such behavioral building blocks (Markowitz et al., 2018).

MoSeq analyzes videos of behaving mice and extracts motifs or ‘syllables’ from their behavior in an unsupervised machine learning approach. These syllables correspond to behavioral snippets such as, for example, rear, scrunch, run, lunge or dive. The researchers have now upgraded their motion vision system to achieve higher spatial and temporal resolution. This increased precision does not change the extracted behavioral syllables or grammar, which has instilled confidence in the validity of the approach. However, “it is not necessarily so that even though we observe that structure, we will be able to find neural correlates for that structure,” says Datta.

To get to the neural correlates that underlie behavioral syllables, the researchers combined MoSeq with two-color fiber photometry to record neural activity in the dorsolateral striatum of mice. They expressed the green and red calcium indicators GCaMP6s and jRCaMP1b in spiny projection neurons of the direct and indirect pathways, respectively. Because there was little optical cross-talk, the researchers could separate the signals well in their two-channel photometry system, and they also accounted for the temporal delay within the optical fiber. To accommodate both the optical fiber and the tether on the mouse’s head, the researchers modified MoSeq so that obscured animal parts could be compensated for. They achieved this by using a generative model to infer missing animal poses.

When analyzing the neural activity with respect to the behavioral syllables, the researchers observed that the activity in both the direct and the indirect pathways changed shortly before a new syllable arose. Furthermore, each behavioral syllable was associated with characteristic waveforms in both pathways. These neural activity patterns were predictive of the behavioral syllables, in particular when the waveforms of both pathways were taken into account.

A similar observation was made by the group of Guohong Cui at the US National Institutes of Health (Meng et al., 2018). They also recorded from the direct and indirect pathways by using spectral unmixing to separate GCaMP6f and jRGECO1a signals during fiber photometry. They conducted their behavioral analysis at lower resolution, but nevertheless found that turning behavior showed stronger correlations with the activity of both pathways than with the activity in individual pathways.

In addition to encoding individual behavioral syllables, the striatum also influences behavioral sequences, or ‘grammar’. Datta and his team demonstrated this by lesioning the dorsolateral striatum. They observed that the syllables were invariant, yet their frequency and sequencing were altered; in other words, the behavioral grammar changed.

“The fact that we can directly observe the neural correlates for syllables and grammar really validates the segmentation of behavior that’s provided by MoSeq,” concludes Datta. MoSeq may result in an imperfect representation of behavior, but the result “gives us confidence . . . that we are heading in the right direction,” says Datta. He now plans to extend the approach to other brain regions and to analyze different behaviors, in different environments.