Article | Published:

Flexible timing by temporal scaling of cortical responses

Nature Neurosciencevolume 21pages102110 (2018) | Download Citation

Abstract

Musicians can perform at different tempos, speakers can control the cadence of their speech, and children can flexibly vary their temporal expectations of events. To understand the neural basis of such flexibility, we recorded from the medial frontal cortex of nonhuman primates trained to produce different time intervals with different effectors. Neural responses were heterogeneous, nonlinear, and complex, and they exhibited a remarkable form of temporal invariance: firing rate profiles were temporally scaled to match the produced intervals. Recording from downstream neurons in the caudate and from thalamic neurons projecting to the medial frontal cortex indicated that this phenomenon originates within cortical networks. Recurrent neural network models trained to perform the task revealed that temporal scaling emerges from nonlinearities in the network and that the degree of scaling is controlled by the strength of external input. These findings demonstrate a simple and general mechanism for conferring temporal flexibility upon sensorimotor and cognitive functions.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Stuphorn, V. & Schall, J. D. Executive control of countermanding saccades by the supplementary eye field. Nat. Neurosci. 9, 925–931 (2006).

  2. 2.

    Kunimatsu, J. & Tanaka, M. Alteration of the timing of self-initiated but not reactive saccades by electrical stimulation in the supplementary eye field. Eur. J. Neurosci. 36, 3258–3268 (2012).

  3. 3.

    Fried, I. et al. Functional organization of human supplementary motor cortex studied by electrical stimulation. J. Neurosci. 11, 3656–3666 (1991).

  4. 4.

    Lewis, P. A., Wing, A. M., Pope, P. A., Praamstra, P. & Miall, R. C. Brain activity correlates differentially with increasing temporal complexity of rhythms during initialisation, synchronisation, and continuation phases of paced finger tapping. Neuropsychologia 42, 1301–1312 (2004).

  5. 5.

    Shima, K. & Tanji, J. Neuronal activity in the supplementary and presupplementary motor areas for temporal organization of multiple movements. J. Neurophysiol. 84, 2148–2160 (2000).

  6. 6.

    Isoda, M. & Hikosaka, O. Switching from automatic to controlled action by monkey medial frontal cortex. Nat. Neurosci. 10, 240–248 (2007).

  7. 7.

    Lu, X., Matsuzawa, M. & Hikosaka, O. A neural correlate of oculomotor sequences in supplementary eye field. Neuron 34, 317–325 (2002).

  8. 8.

    Mello, G. B. M., Soares, S. & Paton, J. J. A scalable population code for time in the striatum. Curr. Biol. 25, 1113–1122 (2015).

  9. 9.

    Gouvêa, T. S. et al. Striatal dynamics explain duration judgments. eLife 4, e11386 (2015).

  10. 10.

    Jin, D. Z., Fujii, N. & Graybiel, A. M. Neural representation of time in cortico-basal ganglia circuits. Proc. Natl. Acad. Sci. USA 106, 19156–19161 (2009).

  11. 11.

    Matell, M. S., Meck, W. H. & Nicolelis, M. A. L. Interval timing and the encoding of signal duration by ensembles of cortical and striatal neurons. Behav. Neurosci. 117, 760–773 (2003).

  12. 12.

    Merchant, H., Harrington, D. L. & Meck, W. H. Neural basis of the perception and estimation of time. Annu. Rev. Neurosci. 36, 313–336 (2013).

  13. 13.

    Mauk, M. D. & Buonomano, D. V. The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 307–340 (2004).

  14. 14.

    Bartolo, R., Prado, L. & Merchant, H. Information processing in the primate basal ganglia during sensory-guided and internally driven rhythmic tapping. J. Neurosci. 34, 3910–3923 (2014).

  15. 15.

    Schultz, W. & Romo, R. Neuronal activity in the monkey striatum during the initiation of movements. Exp. Brain Res. 71, 431–436 (1988).

  16. 16.

    Tanaka, M. Inactivation of the central thalamus delays self-timed saccades. Nat. Neurosci. 9, 20–22 (2006).

  17. 17.

    Treisman, M. Temporal discrimination and the indifference interval. Implications for a model of the “internal clock”. Psychol. Monogr. 77, 1–31 (1963).

  18. 18.

    Killeen, P. R. & Fetterman, J. G. A behavioral theory of timing. Psychol. Rev. 95, 274–295 (1988).

  19. 19.

    Matell, M. S. & Meck, W. H. Cortico-striatal circuits and interval timing: coincidence detection of oscillatory processes. Brain Res. Cogn. Brain Res. 21, 139–170 (2004).

  20. 20.

    Karmarkar, U. R. & Buonomano, D. V. Timing in the absence of clocks: encoding time in neural network states. Neuron 53, 427–438 (2007).

  21. 21.

    Buonomano, D. V. & Laje, R. Population clocks: motor timing with neural dynamics. Trends Cogn. Sci. 14, 520–527 (2010).

  22. 22.

    Brody, C. D., Hernández, A., Zainos, A. & Romo, R. Timing and neural encoding of somatosensory parametric working memory in macaque prefrontal cortex. Cereb. Cortex 13, 1196–1207 (2003).

  23. 23.

    Komura, Y. et al. Retrospective and prospective coding for predicted reward in the sensory thalamus. Nature 412, 546–549 (2001).

  24. 24.

    Merchant, H., Zarco, W., Pérez, O., Prado, L. & Bartolo, R. Measuring time with different neural chronometers during a synchronization-continuation task. Proc. Natl. Acad. Sci. USA 108, 19784–19789 (2011).

  25. 25.

    Emmons, E. B. et al. Rodent medial frontal control of temporal processing in the dorsomedial striatum. J. Neurosci. 37, 8718–8733 (2017).

  26. 26.

    Gibbon, J. Scalar expectancy theory and Weber’s law in animal timing. Psychol. Rev. 84, 279 (1977).

  27. 27.

    Rakitin, B. C. et al. Scalar expectancy theory and peak-interval timing in humans. J. Exp. Psychol. Anim. Behav. Process. 24, 15–33 (1998).

  28. 28.

    Creelman, C. D. Human discrimination of auditory duration. J. Acoust. Soc. Am. 34, 582–593 (1962).

  29. 29.

    Gibbon, J., Church, R. M. & Meck, W. H. Scalar timing in memory. Ann. NY Acad. Sci. 423, 52–77 (1984).

  30. 30.

    Grondin, S. From physical time to the first and second moments of psychological time. Psychol. Bull. 127, 22–44 (2001).

  31. 31.

    Hikosaka, O., Sakamoto, M. & Usui, S. Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. J. Neurophysiol. 61, 780–798 (1989).

  32. 32.

    Parthasarathy, H. B., Schall, J. D. & Graybiel, A. M. Distributed but convergent ordering of corticostriatal projections: analysis of the frontal eye field and the supplementary eye field in the macaque monkey. J. Neurosci. 12, 4468–4488 (1992).

  33. 33.

    Sussillo, D., Churchland, M. M., Kaufman, M. T. & Shenoy, K. V. A neural network that finds a naturalistic solution for the production of muscle activity. Nat. Neurosci. 18, 1025–1033 (2015).

  34. 34.

    Li, N., Daie, K., Svoboda, K. & Druckmann, S. Robust neuronal dynamics in premotor cortex during motor planning. Nature 532, 459–464 (2016).

  35. 35.

    Mante, V., Sussillo, D., Shenoy, K. V. & Newsome, W. T. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013).

  36. 36.

    Hanes, D. P. & Schall, J. D. Neural control of voluntary movement initiation. Science 274, 427–430 (1996).

  37. 37.

    Sussillo, D. & Barak, O. Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural Comput. 25, 626–649 (2013).

  38. 38.

    Machens, C. K., Romo, R. & Brody, C. D. Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 307, 1121–1124 (2005).

  39. 39.

    Hanks, T., Kiani, R. & Shadlen, M. N. A neural mechanism of speed-accuracy tradeoff in macaque area LIP. eLife 3, e02260 (2014).

  40. 40.

    Wang, X.-J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).

  41. 41.

    Meck, W. H. Neuropharmacology of timing and time perception. Brain Res. Cogn. Brain Res. 3, 227–242 (1996).

  42. 42.

    Jazayeri, M. & Shadlen, M. N. A neural mechanism for sensing and reproducing a time interval. Curr. Biol. 25, 2599–2609 (2015).

  43. 43.

    Buhusi, C. V. & Meck, W. H. Differential effects of methamphetamine and haloperidol on the control of an internal clock. Behav. Neurosci. 116, 291–297 (2002).

  44. 44.

    Soares, S., Atallah, B. V. & Paton, J. J. Midbrain dopamine neurons control judgment of time. Science 354, 1273–1277 (2016).

  45. 45.

    Chaudhuri, R., Knoblauch, K., Gariel, M.-A., Kennedy, H. & Wang, X.-J. A large-scale circuit mechanism for hierarchical dynamical processing in the primate cortex. Neuron 88, 419–431 (2015).

  46. 46.

    Fujii, N., Mushiake, H. & Tanji, J. Distribution of eye- and arm-movement-related neuronal activity in the SEF and in the SMA and Pre-SMA of monkeys. J. Neurophysiol. 87, 2158–2166 (2002).

  47. 47.

    Namboodiri, V. M. & Hussain Shuler, M. G. Report of interval timing or action? Proc. Natl. Acad. Sci. USA 111, E2239–E2239 (2014).

  48. 48.

    Xu, M., Zhang, S.-Y., Dan, Y. & Poo, M.-M. Representation of interval timing by temporally scalable firing patterns in rat prefrontal cortex. Proc. Natl. Acad. Sci. USA 111, 480–485 (2013).

Download references

Acknowledgements

We thank M.S. Fee, J.J. DiCarlo, and R. Desimone for comments on the manuscript, and we thank D. Sussillo for advice on modeling. D.N. was supported by the Rubicon Grant (2015/446-14-008) from the Netherlands Scientific Organization (NWO). M.J. is supported by the NIH (NINDS-NS078127), the Sloan Foundation, the Klingenstein Foundation, the Simons Foundation, the Center for Sensorimotor Neural Engineering, and the McGovern Institute.

Author information

Author notes

    • Jing Wang
    • , Eghbal A. Hosseini
    •  & Mehrdad Jazayeri

    Present address: Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA

  1. Jing Wang and Devika Narain contributed equally to this research.

Affiliations

  1. McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA

    • Jing Wang
    • , Devika Narain
    •  & Mehrdad Jazayeri
  2. Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA

    • Devika Narain
    • , Eghbal A. Hosseini
    •  & Mehrdad Jazayeri
  3. Netherlands Institute for Neuroscience, Amsterdam, The Netherlands

    • Devika Narain
  4. Department of Neuroscience, Erasmus Medical Center, Rotterdam, The Netherlands

    • Devika Narain

Authors

  1. Search for Jing Wang in:

  2. Search for Devika Narain in:

  3. Search for Eghbal A. Hosseini in:

  4. Search for Mehrdad Jazayeri in:

Contributions

J.W. was responsible for all aspects of experiments and analyses and developed the simplified model. D.N. was responsible for the development of the recurrent neural network model. E.A.H. helped with the data collection and analysis. M.J. was responsible for all aspects of the project. All authors helped with the interpretation of data and writing the paper.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Mehrdad Jazayeri.

Integrated Supplementary Information

  1. Supplementary Figure 1 Scaling index (SI) of single neurons in different brain areas

    (a) Histograms on the top show the normalized distribution of the scaling index for individual neurons in the MFC (n = 416 neurons from both animals), caudate (n = 278 neurons), and thalamus (n = 846 neurons) for the Eye (Left) and Hand conditions (Right). The bottom panel shows a comparison of the cumulative probability distribution of scaling index across the three areas. The thalamus shows a predominance of smaller scaling index value (Mann-Whitney-Wilcoxon test, one-tailed, W(1260) = 310,733, z = 7.89, *** P < .001 in comparison with MFC, and W(1122) = 189,163, z = 6.98, *** P < .001 in comparison with caudate) (b) Example PSTHs covering a range of SIs in the three brain areas. The SI value and effector condition for each neuron is indicated.

  2. Supplementary Figure 2 The time course of PCs (left) and SCs (right) for MFC data

    First 9 PCs over the course of the production interval (abscissa) that explain 80% of variance in the MFC data (n = 281 neurons for Monkey A) in decreasing order of variance explained (left) for Short (warm colors) and Long (cool colors) intervals. First 9 SCs, obtained for the same data, in decreasing order of scaling (right, see Methods).

  3. Supplementary Figure 3 Analysis of scaling with surrogate data

    (a) Venn diagram showing the various constraints considered for non-scaling models. All surrogate data was generated from a Gaussian Process (GP) with the same level of temporal smoothness (white rectangle, S) as the data. We considered three additional constraints to make the surrogate data more similar to neural data without an explicit requirement for scaling. One constraint required responses for all production intervals to be at the same level at the time of Set and at the time of response. We refer to this constraint as endpoint matching (red circle, E). Another constraint required that the dimensionality of the surrogate data match the neural data, and additionally the variance explained by each principal component (PC) be matched. We refer to this constraint as dimensionality matching (green circle, D). Finally, we considered a constraint that required the collection of responses for different production intervals to have the same correlation (quantified as R2) as expected from perfect scaling. We refer to this constraint as correlation matching (blue circle, C). We created surrogate data for each constraint and for various combination of constraints, and compared the scaling properties to the original data. Note that each constraint characterized a superset of the scaling hypothesis. (b) Example traces showing the procedure for generating the surrogate data (n = 281 surrogate units) in the C+D+E+S model for 5 randomly selected surrogate units aligned to the time of Set. We first sampled a Short trace (red) from a Gaussian process. The trace in blue corresponds to the perfectly scaled version of the red trace and is not a sample from the surrogate model. The surrogate data were generated using a constrained Gaussian Process (GP) prior as follows: the response for the shortest production intervals (red) was sampled from a GP with the same level of temporal smoothness as the neural data. The corresponding response with perfect scaling was generated by linear scaling (shown in blue). Note that the trace in blue is not a sample from the GP and is therefore, not part of the surrogate data. The gray traces correspond to the surrogate data. To generate the surrogate data, we drew samples from the Gaussian process that satisfied several criteria. First, the starting point as well as the ending point of every gray trace had to be perfectly matched to the starting point and ending point of the perfectly scaled blue trace. Second, across the population of surrogate data, the dimensionality had to match observed neural data. Finally, the correlation between every gray trace and the red trace was the same as the correlation between the red and blue trace. In this way, every sample of GP (gray traces) matched the smoothness, endpoints, dimensionality and correlation as the real data (i.e., C+D+E+S model). (c) Cumulative percentage variance explained by PCs and SCs for the surrogate data generated from the non-scaling C+D+E+S model. (d) The first 9 principal components of population activity (PC, left) and the corresponding 9 scaling components (right, SCs) plotted as the function of time from Set for the non-scaling C+D+E+S model. Note that PCs and SCs are based on the surrogate data (gray traces in panel b) – not the perfectly scaled data (blue traces in panel b).

  4. Supplementary Figure 4 Relationship between MFC population activity and behavior

    The speed of neural trajectory in MFC (n = 281 neurons for Monkey A, n = 135 neurons for Monkey D) within the scaling subspace spanned by the first 3 SCs predicted Tp across both Short and Long conditions on a trial-by-trial basis. Both ordinate and abscissa follow a logarithmic scale. The case of hand trials for Monkey A was shown in Fig. 5d. Here, all the other conditions are shown.

  5. Supplementary Figure 5 Analysis of scaling at the population level in the caudate (top) and thalamus (bottom)

    Left column: Population activity (n = 101 neurons for caudate and n = 481 neurons for thalamus, Moneky A) profiles projected onto the first 3 principal components (PCs). Activity profiles associated with different produced intervals for Short and Long conditions are plotted in different colors (same color scheme used throughout the paper). The state at 700 ms after Set is shown along the trajectories (diamond). Second column from the left: Population activity projected onto the first 3 scaling components (SCs). Activity spanned by the first 3 SCs overlap for different intervals in the caudate but not in the thalamus. Third column from left: Variance explained for individual SCs as a function of scaling index. Right column: The speed of neural trajectory within the scaling subspace spanned by the first 3 SCs. Both ordinate and abscissa follow a logarithmic scale.

  6. Supplementary Figure 6 Alternative recurrent neural network models

    (a) A recurrent neural network (RNN) trained to use an interval-dependent Cue input to produce time intervals flexibly. The network was trained using a non-scaling exponential objective with a fixed time constant. (b) The speed of dynamics measured within the space spanned by the first three PCs predicted Tp across both Short and Long conditions. Both ordinate and abscissa follow a logarithmic scale. (c) The response profiles of randomly selected units in the network (a) aligned to the time of Set. (d) A RNN trained to use a brief pulse as the interval-dependent Cue input to produce time intervals flexibly. The network was trained using a linear ramping objective like the network in the main text. (e) The speed of dynamics predicted Tp across both Short and Long conditions with the same format as b). (f) The response profiles of randomly selected units in network (d) aligned to the time of Set.

  7. Supplementary Figure 7 Temporal scaling in the recurrent network mode

    (a) The first 9 scaling components (SCs) of the population activity in the RNN with a scaling output function (Fig. 5). The early SCs correspond to the recurrent subspace, and the last SC represents the input subspace. (b) The average speed of population activity in the subspace spanned by the first 3 SCs is predictive of both within context and across context variations in Tp. Both ordinate and abscissa follow a logarithmic scale. (c) The average firing rate of the population activity projected onto the last SC is also predictive of Tp. (d) Cumulative percentage variance explained by PCs (white) and SCs (black). The dashed vertical line corresponds to the 9th component. (e-h) Same as a-d, for a network that was trained for a non-scaling exponential output objective function (Supplementary Fig. 6d).

  8. Supplementary Figure 8 Nonscaling population activity in MFC

    Left:  The time course of the SC9 (the least scaling component) across conditions. Right: The average firing rate of population activity (n = 281 neurons for Monkey A) projected onto SC9 (left), also the putative input subspace, increases with produced intervals (Tp). This is consistent with the hypothesis that the average firing rate in the non-scaling subspace controls speed. Based on the recurrent network model, this subspace likely reflects the input drive to MFC. Both ordinate and abscissa are plotted in a logarithmic scale.

Supplementary information

  1. Supplementary Text and Figures

    Supplementary Figures 1–7 and Supplementary Note

  2. Life Sciences Reporting Summary

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/s41593-017-0028-6

Further reading