How MT cells analyze the motion of visual patterns


Neurons in area MT (V5) are selective for the direction of visual motion. In addition, many are selective for the motion of complex patterns independent of the orientation of their components, a behavior not seen in earlier visual areas. We show that the responses of MT cells can be captured by a linear-nonlinear model that operates not on the visual stimulus, but on the afferent responses of a population of nonlinear V1 cells. We fit this cascade model to responses of individual MT neurons and show that it robustly predicts the separately measured responses to gratings and plaids. The model captures the full range of pattern motion selectivity found in MT. Cells that signal pattern motion are distinguished by having convergent excitatory input from V1 cells with a wide range of preferred directions, strong motion opponent suppression and a tuned normalization that may reflect suppressive input from the surround of V1 cells.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Figure 1: Component and pattern MT cell responses to plaid stimuli.
Figure 2: Quantification of neuronal responses to plaids.
Figure 3: The cascade model and its characterization.
Figure 4: Comparison of actual and model-predicted responses to plaids for five example cells.
Figure 5: Comparison of the pattern index observed and predicted by the cascade model.
Figure 6: Recovered models for five example MT neurons, corresponding to the same cells whose data are shown in Figure 4.
Figure 7: Relationship between the recovered cascade model parameters and pattern index.
Figure 8: Dissection of the elements of the cascade model that create particular kinds of selectivity for the motion of plaids.


  1. 1

    Felleman, D.J. & Van Essen, D.C. Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex 1, 1–47 (1991).

  2. 2

    Movshon, J.A., Adelson, E.H., Gizzi, M.S. & Newsome, W.T. The analysis of moving visual patterns. in Pattern Recognition Mechanisms (Pontificiae Academiae Scientiarum Scripta Varia) Vol. 54 (eds. Chagas, C., Gattass, R. & Gross, C.) 117–151 (Vatican Press, Rome, 1985).

  3. 3

    Simoncelli, E.P. & Heeger, D.J. A model of neuronal responses in visual area MT. Vision Res. 38, 743–761 (1998).

  4. 4

    Adelson, E.H. & Bergen, J.R. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A 2, 284–299 (1985).

  5. 5

    Hubel, D.H. & Wiesel, T.N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. (Lond.) 160, 106–154 (1962).

  6. 6

    DeAngelis, G.C., Ohzawa, I. & Freeman, R.D. Spatiotemporal organization of simple-cell receptive fields in the cat's striate cortex. I. General characteristics and postnatal development. J. Neurophysiol. 69, 1091–1117 (1993).

  7. 7

    Emerson, R.C., Bergen, J.R. & Adelson, E.H. Directionally selective complex cells and the computation of motion energy in cat visual cortex. Vision Res. 32, 203–218 (1992).

  8. 8

    Adelson, E.H. & Movshon, J.A. Phenomenal coherence of moving visual patterns. Nature 300, 523–525 (1982).

  9. 9

    Zeki, S.M. Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey. J. Physiol. (Lond.) 236, 549–573 (1974).

  10. 10

    Movshon, J.A. & Newsome, W.T. Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys. J. Neurosci. 16, 7733–7741 (1996).

  11. 11

    Carandini, M. et al. Do we know what the early visual system does? J. Neurosci. 25, 10577–10597 (2005).

  12. 12

    Movshon, J.A., Thompson, I.D. & Tolhurst, D.J. Spatial summation in the receptive fields of simple cells in the cat's striate cortex. J. Physiol. (Lond.) 283, 53–77 (1978).

  13. 13

    Movshon, J.A., Thompson, I.D. & Tolhurst, D.J. Receptive field organization of complex cells in the cat's striate cortex. J. Physiol. (Lond.) 283, 79–99 (1978).

  14. 14

    Jones, J.P., Stepnoski, A. & Palmer, L.A. The two-dimensional spectral structure of simple receptive fields in cat striate cortex. J. Neurophysiol. 58, 1212–1232 (1987).

  15. 15

    Mante, V. Testing Models of Cortical Area MT. Thesis, Institute of Neuroinformatics, ETH, University of Zurich (2000).

  16. 16

    Smith, M.A., Majaj, N.J. & Movshon, J.A. Dynamics of motion signaling by neurons in macaque area MT. Nat. Neurosci. 8, 220–228 (2005).

  17. 17

    Carandini, M., Heeger, D.J. & Movshon, J.A. Linearity and normalization in simple cells of the macaque primary visual cortex. J. Neurosci. 17, 8621–8644 (1997).

  18. 18

    Lennie, P. & Movshon, J.A. Coding of color and form in the geniculostriate visual pathway. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 22, 2013–2033 (2005).

  19. 19

    Ringach, D.L., Sapiro, G. & Shapley, R. A subspace reverse-correlation technique for the study of visual neurons. Vision Res. 37, 2455–2464 (1997).

  20. 20

    Theunissen, F.E., Sen, K. & Doupe, A.J. Spectral-temporal receptive fields of nonlinear auditory neurons obtained using natural sounds. J. Neurosci. 20, 2315–2331 (2000).

  21. 21

    David, S.V., Vinje, W.E. & Gallant, J.L. Natural stimulus statistics alter the receptive field structure of V1 neurons. J. Neurosci. 24, 6991–7006 (2004).

  22. 22

    Maunsell, J.H. & van Essen, D.C. The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. J. Neurosci. 3, 2563–2586 (1983).

  23. 23

    Sincich, L.C., Park, K.F., Wohlgemuth, M.J. & Horton, J.C. Bypassing V1: a direct geniculate input to area MT. Nat. Neurosci. 7, 1123–1128 (2004).

  24. 24

    Levitt, J.B., Kiper, D.C. & Movshon, J.A. Receptive fields and functional architecture of macaque V2. J. Neurophysiol. 71, 2517–2542 (1994).

  25. 25

    Gegenfurtner, K.R., Kiper, D.C. & Levitt, J.B. Functional properties of neurons in macaque area V3. J. Neurophysiol. 77, 1906–1923 (1997).

  26. 26

    Rodman, H.R., Gross, C.G. & Albright, T.D. Afferent basis of visual response properties in area MT of the macaque. I. Effects of striate cortex removal. J. Neurosci. 9, 2033–2050 (1989).

  27. 27

    Collins, C.E., Lyon, D.C. & Kaas, J.H. Responses of neurons in the middle temporal visual area after long-standing lesions of the primary visual cortex in adult new world monkeys. J. Neurosci. 23, 2251–2264 (2003).

  28. 28

    Heeger, D.J. Normalization of cell responses in cat striate cortex. Vis. Neurosci. 9, 181–197 (1992).

  29. 29

    Geisler, W.S. & Albrecht, D.G. Cortical neurons: isolation of contrast gain control. Vision Res. 32, 1409–1410 (1992).

  30. 30

    Albrecht, D.G. & Hamilton, D.B. Striate cortex of monkey and cat: contrast response function. J. Neurophysiol. 48, 217–237 (1982).

  31. 31

    Cavanaugh, J.R., Bair, W. & Movshon, J.A. Selectivity and spatial distribution of signals from the receptive field surround in macaque V1 neurons. J. Neurophysiol. 88, 2547–2556 (2002).

  32. 32

    Cavanaugh, J.R., Bair, W. & Movshon, J.A. Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. J. Neurophysiol. 88, 2530–2546 (2002).

  33. 33

    Sceniak, M.P., Hawken, M.J. & Shapley, R. Visual spatial characterization of macaque V1 neurons. J. Neurophysiol. 85, 1873–1887 (2001).

  34. 34

    Livingstone, M.S., Pack, C.C. & Born, R.T. Two-dimensional substructure of MT receptive fields. Neuron 30, 781–793 (2001).

  35. 35

    Pack, C.C., Livingstone, M.S., Duffy, K.R. & Born, R.T. End-stopping and the aperture problem: two-dimensional motion signals in macaque V1. Neuron 39, 671–680 (2003).

  36. 36

    Pack, C.C., Gartland, A.J. & Born, R.T. Integration of contour and terminator signals in visual area MT of alert macaque. J. Neurosci. 24, 3268–3280 (2004).

  37. 37

    van den Berg, A.V. & Noest, A.J. Motion transparency and coherence in plaids: the role of end-stopped cells. Exp. Brain Res. 96, 519–533 (1993).

  38. 38

    Tinsley, C.J. et al. The nature of V1 neural responses to 2D moving patterns depends on receptive-field structure in the marmoset monkey. J. Neurophysiol. 90, 930–937 (2003).

  39. 39

    Rockland, K.S. Bistratified distribution of terminal arbors of individual axons projecting from area V1 to middle temporal area (MT) in the macaque monkey. Vis. Neurosci. 3, 155–170 (1989).

  40. 40

    Kohn, A. & Movshon, J.A. Neuronal adaptation to visual motion in area MT of the macaque. Neuron 39, 681–691 (2003).

  41. 41

    Bair, W. & Movshon, J.A. Adaptive temporal integration of motion in direction-selective neurons in macaque visual cortex. J. Neurosci. 24, 7305–7323 (2004).

  42. 42

    Sclar, G., Maunsell, J.H. & Lennie, P. Coding of image contrast in central visual pathways of the macaque monkey. Vision Res. 30, 1–10 (1990).

  43. 43

    Britten, K.H. & Heuer, H.W. Spatial summation in the receptive fields of MT neurons. J. Neurosci. 19, 5074–5084 (1999).

  44. 44

    Rodman, H.R. & Albright, T.D. Coding of visual stimulus velocity in area MT of the macaque. Vision Res. 27, 2035–2048 (1987).

  45. 45

    Rodman, H.R. & Albright, T.D. Single-unit analysis of pattern-motion selective properties in the middle temporal visual area (MT). Exp. Brain Res. 75, 53–64 (1989).

  46. 46

    Chance, F.S., Nelson, S.B. & Abbott, L.F. Complex cells as cortically amplified simple cells. Nat. Neurosci. 2, 277–282 (1999).

  47. 47

    Priebe, N.J., Mechler, F., Carandini, M. & Ferster, D. The contribution of spike threshold to the dichotomy of cortical simple and complex cells. Nat. Neurosci. 7, 1113–1122 (2004).

  48. 48

    Rust, N.C., Schwartz, O., Movshon, J.A. & Simoncelli, E.P. Spatiotemporal elements of macaque V1 receptive fields. Neuron 46, 945–956 (2005).

  49. 49

    Efron, B. & Tibshirani, R.J. An Introduction to the Boostrap (Chapman & Hall/CRC Press, Boca Raton, Florida, 1993).

  50. 50

    Paninski, L. Maximum likelihood estimation of cascade point-process neural encoding models. Network 15, 243–262 (2004).

Download references


We are grateful to M. Carandini and N. Majaj for helpful discussions. This work was supported by the Howard Hughes Medical Institute through an Investigatorship to E.P.S. and by a grant from the National Eye Institute to J.A.M. (EY02017). V.M. was supported by a grant to M. Carandini from the Swiss National Science Foundation.

Author information

Correspondence to Nicole C Rust.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Rust, N., Mante, V., Simoncelli, E. et al. How MT cells analyze the motion of visual patterns. Nat Neurosci 9, 1421–1431 (2006).

Download citation

Further reading