Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Cognitive neuroscience

Neural mechanisms for the recognition of biological movements

Key Points

  • Humans can recognize biological movements, such as walking, accurately and robustly. This review uses a neurophysiologically plausible and quantitative model as a tool for organizing and making sense of the available experimental data, despite its growing size and complexity.

  • Most experimental results can be accounted for by simple neural mechanisms, under the two key assumptions that recognition is based on a hierarchical feedforward cortical architecture and learned prototypical patterns. Such prototypes might be stored in specific neurons in the visual system.

  • The model shows that recognition of biological movements can be achieved with plausible neural mechanisms in a way that is quantitatively consistent with the experimental data on pattern selectivity, view dependence and robustness of recognition.

  • The model comprises two parallel pathways, one corresponding to the dorsal pathway (specialized for the analysis of motion information) and one to the ventral pathway (specialized for the analysis of form information). In each pathway, neural feature detectors extract form or optic-flow features with increasing complexity along the hierarchy. The position and size invariance of the feature detectors also increases along the hierarchy. Experimental data and quantitative simulations indicate that the ventral and dorsal pathways are both needed for the recognition of normal biological movement stimuli, whereas the recognition of point-light stimuli seems mainly to depend on the dorsal pathway.

  • The proposed architecture predicts the existence of neurons that can learn to respond selectively to new biological movement patterns. It also predicts that arbitrary complex movement patterns should be learnable, as long as they provide suitable stimulation of the mid- and low-level feature detectors of the two pathways.

  • The model predicts the existence of neurons in the dorsal pathway that become selectively activated by complex optic-flow patterns that arise for biological movement patterns.

  • It is demonstrated that attention and top–down influences are not required to solve the basic tasks of motion recognition. These factors may be necessary for more sophisticated motion recognition tasks. The model cannot account for such influences of attention and of different tasks.

  • A number of open questions and predictions of the model are considered. The use of a quantitative model allows us to generate specific predictions and to show that a neurophysiologically consistent, learning-based, feedforward model can reproduce many key experimental results. Open questions include how information from the two pathways is integrated, and which neural mechanisms underlie sequence selectivity in both pathways.

Abstract

The visual recognition of complex movements and actions is crucial for the survival of many species. It is important not only for communication and recognition at a distance, but also for the learning of complex motor actions by imitation. Movement recognition has been studied in psychophysical, neurophysiological and imaging experiments, and several cortical areas involved in it have been identified. We use a neurophysiologically plausible and quantitative model as a tool for organizing and making sense of the experimental data, despite their growing size and complexity. We review the main experimental findings and discuss possible neural mechanisms, and show that a learning-based, feedforward model provides a neurophysiologically plausible and consistent summary of many key experimental results.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Hierarchical neural model that provides a unifying interpretation of the data.
Figure 2: Neural mechanisms of sequence selectivity.
Figure 3: Testing selectivity and generalization.
Figure 4: View variance and robustness.
Figure 5: Prediction of neural activities for a comparison with functional imaging experiments.

Similar content being viewed by others

References

  1. Ewert, J. -P. Neuroethology of releasing mechanisms: Prey-catching in toads. Behav. Brain Sci. 10, 337–405 (1987).

    Article  Google Scholar 

  2. Morris, D. The reproductive behaviour of the zebra finch (Poephilla guttata), with special reference to pseudofemale behaviour. Behaviour 6, 271–322 (1954).

    Article  Google Scholar 

  3. Darwin, C. The Expressions of the Emotions in Man and Animals (Appleton, New York, 1872).

    Book  Google Scholar 

  4. Andrew, R. J. Evolution of facial expression. Science 142, 1034–1041 (1963).

    Article  CAS  PubMed  Google Scholar 

  5. Adolphs, R. The neurobiology of social cognition. Curr. Opin. Neurobiol. 11, 231–239 (2001).

    Article  CAS  PubMed  Google Scholar 

  6. Johansson, G. Visual perception of biological motion and a model for its analysis. Percept. Psychophys. 14, 201–211 (1973). A classical paper reporting the original experiment in which subjects were able to recognize complex actions from point-light displays that were generated by filming moving actors.

    Article  Google Scholar 

  7. Bruce, C., Desimone, R. & Gross, C. G. Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J. Neurophysiol. 46, 369–384 (1981).

    Article  CAS  PubMed  Google Scholar 

  8. Perrett, D. I. et al. Visual analysis of body movements by neurones in the temporal cortex in the macaque monkey: a preliminary report. Behav. Brain Res. 16, 153–170 (1985).

    Article  CAS  PubMed  Google Scholar 

  9. Perrett, D. I. et al. in AI and the Eye (eds Blake, A. & Troscianko, T.) 181–200 (John Wiley, Chichester, 1990).

    Google Scholar 

  10. Oram, M. W. & Perrett, D. I. Responses of anterior superior temporal polysensory (STPa) neurons to 'biological motion' stimuli. J. Cogn. Neurosci. 6, 99–116 (1994). References 8–10 present electrophysiological studies in macaque monkeys with recordings in different parts of the STS during presentation of biological movement stimuli. In different areas, neurons were found that seemed to respond selectively for biological movements of normal and point-light stimuli.

    Article  CAS  PubMed  Google Scholar 

  11. Jellema, T., Baker, C. I., Wicker, B. & Perrett, D. I. Neural representation for the perception of the intentionality of actions. Brain Cogn. 44, 280–302 (2000).

    Article  CAS  PubMed  Google Scholar 

  12. Perrett, D. I. et al. Frameworks of analysis for the neural representation of animate objects and actions. Neural representation for the perception of the intentionality of actions. J. Exp. Biol. 146, 87–113 (1989).

    Article  CAS  PubMed  Google Scholar 

  13. Rizzolatti, G., Fogassi, L. & Gallese, V. Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Rev. Neurosci. 2, 661–670 (2001). A review article about the neural basis of action recognition and imitation with a special focus on the 'mirror neuron' system.

    Article  CAS  Google Scholar 

  14. Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. Action recognition in the premotor cortex. Brain 119, 593–609 (1996).

    Article  PubMed  Google Scholar 

  15. Rizzolatti, G. & Arbib, M. A. Language within our grasp. Trends Neurosci. 21, 188–194 (1998).

    Article  CAS  PubMed  Google Scholar 

  16. Kohler, E et al. Hearing sounds, understanding actions: action representation in mirror neurons. Science 297, 846–848 (2002).

    Article  CAS  PubMed  Google Scholar 

  17. Decety, J. & Grèzes, J. Neural mechanisms subserving the perception of human actions. Trends Cogn. Sci. 3, 172–178 (1999). A review article about functional imaging studies of the perception of human actions.

    Article  CAS  PubMed  Google Scholar 

  18. Allison, T., Puce, A. & McCarthy, G. Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267–278 (2000).

    Article  CAS  PubMed  Google Scholar 

  19. Grossman, E. D. & Blake, R. Brain areas active during visual perception of biological motion. Neuron 35, 1167–1175 (2002). An fMRI study that compares activity in different form- and motion-selective areas for biological and non-biological motion.

    Article  CAS  PubMed  Google Scholar 

  20. Bonda, E. et al. Specific involvement of human parietal systems and the amygdala in the perception of biological motion J. Neurosci. 16, 3737–3744 (1996). One of the first fMRI studies showing activity that is specific for biological motion in the human STS.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Howard, R. J. et al. A direct demonstration of functional specialization within motion-related visual and auditory cortex of the human brain. Curr. Biol. 6, 1015–1019 (1996).

    Article  CAS  PubMed  Google Scholar 

  22. Grossman, E, et al. Brain areas involved in perception of biological motion. J. Cogn. Neurosci. 12, 711–720 (2000).

    Article  CAS  PubMed  Google Scholar 

  23. Vaina, L. M. et al. Functional neuroanatomy of biological motion perception in humans. Proc. Natl Acad. Sci USA 98, 11656–11661 (2001). An fMRI study that shows an involvement of the opponent motion-selective area KO in biological motion recognition, and an influence of the task on the identified biological motion-sensitive areas.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Iacoboni, M. et al. Cortical mechanisms of human imitation. Science 286, 2526–2528 (1999).

    Article  CAS  PubMed  Google Scholar 

  25. Buccino, G. et al. Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur. J. Neurosci. 13, 400–404 (2001).

    CAS  PubMed  Google Scholar 

  26. Servos, P., Osu, R., Santi, A. & Kawato, M. The neural substrates of biological motion perception: an fMRI study. Cereb. Cortex 12, 772–782 (2002).

    Article  PubMed  Google Scholar 

  27. Mather, G., Radford, K. & West, S. Low-level visual processing of biological motion. Proc. R. Soc. Lond. B 249, 149–155 (1992).

    Article  CAS  Google Scholar 

  28. Bobick, A. F. Movement, activity, and action: the role of knowledge in the perception of motion. Phil. Trans. R. Soc. Lond. B 352, 1257–1265 (1997).

    Article  CAS  Google Scholar 

  29. Todd, J. T. Perception of gait. J. Exp. Psychol. Hum. Percept. Perform. 9, 31–42 (1993).

    Article  Google Scholar 

  30. Beintema, J. A. & Lappe, M. Perception of biological motion without local image motion. Poc. Natl Acad. Sci. USA 99, 5661–5663 (2002).

    Article  CAS  Google Scholar 

  31. Downing, P., Jiang, Y., Shuman, M. & Kanwisher, N. A cortical area selective for visual processing of the human body. Science 293, 2470–2473 (2001).

    Article  CAS  PubMed  Google Scholar 

  32. Pollick, F. E., Fidopiastis, C. & Braden, V. Recognizing the style of spatially exaggerated tennis serves. Perception 30, 323–338 (2001). A psychophysical experiment using morphing to generate exaggerations of biological movements. The recognition of spatially exaggerated patterns is improved.

    Article  CAS  PubMed  Google Scholar 

  33. Hill, H. & Johnston, A. Categorizing sex and identity from the biological motion of faces. Curr. Biol. 11, 880–885 (2001). A psychophysical study showing that identity can be recognized from facial movements even when identity-specific form information is removed from the face.

    Article  CAS  PubMed  Google Scholar 

  34. Tarr, M. J. & Bülthoff, H. H. Image-based object recognition in man, monkey and machine. Cognition 67, 1–20 (1998). A review article about view-based object recognition in biological and technical systems.

    Article  CAS  PubMed  Google Scholar 

  35. Poggio, T. & Edelman, S. A network that learns to recognize three-dimensional objects. Nature 343, 263–266 (1990).

    Article  CAS  PubMed  Google Scholar 

  36. Logothetis, N. K., Pauls, J. & Poggio, T. Shape representation in the inferior temporal cortex of monkeys. Curr. Biol. 5, 552–563 (1995). An electrophysiological study showing that neurons in area IT of monkey cortex can learn to become selective for artificial three-dimensional shapes. This study also provides evidence for the neural encoding of complex three-dimensional shapes in terms of prototypical views and in particular for size and position invariance being independent of specific experience.

    Article  CAS  PubMed  Google Scholar 

  37. Ungerleider, L. G. & Mishkin, M. in Analysis of Visual Behavior (eds Ingle, D. J., Goodale, M. A. & Mansfield, R. J. W.) 549–586 (MIT Press, Cambridge, Massachusetts, 1982).

    Google Scholar 

  38. Felleman, D. J. & van Essen, D. C. Distributed hierarchical processing in the primate visual cortex. Cereb. Cortex 1, 1–47 (1991).

    Article  CAS  PubMed  Google Scholar 

  39. Milner, A. D. & Goodale, M. A. Visual pathways to perception and action. Prog. Brain Res. 95, 317–337 (1993).

    Article  CAS  PubMed  Google Scholar 

  40. Thorpe, S., Fize, D. & Marlot, C. Speed of processing in the human visual system. Nature 381, 520–522 (1996). An electroencephalography study showing that selective cortical activity in a categorization task can be detected as early as 150 ms after stimulus onset. This makes it unlikely that complex feedback circuits are needed for basic, 'immediate' categorization tasks.

    Article  CAS  PubMed  Google Scholar 

  41. Oram, M. W. & Perrett, D. I. Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. J. Neurophysiol. 76, 109–129 (1996).

    Article  CAS  PubMed  Google Scholar 

  42. Johansson, G. Spatio-temporal differentiation and integration in visual motion perception. Psychol. Res. 38, 379–393 (1976).

    Article  CAS  PubMed  Google Scholar 

  43. Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition. Nature Neurosci. 2, 1019–1025 (1999). A neurophysiologically plausible model for the recognition of stationary objects that provides the skeleton of the form pathway of our model for motion recognition. The study shows that pooling by a maximum-like operation provides feature selectivity simultaneously with sufficient position and size invariance.

    Article  CAS  PubMed  Google Scholar 

  44. Riesenhuber, M. & Poggio, T. Models of object recognition. Nature Neurosci. 3, Suppl. 1199–1204 (2000).

    Article  CAS  PubMed  Google Scholar 

  45. Riesenhuber, M. & Poggio, T. Neural mechanisms of object recognition. Curr. Opin. Neurobiol. 12, 162–168 (2002).

    Article  CAS  PubMed  Google Scholar 

  46. Kourtzi, Z. & Kanwisher, N. Activation in human MT/MST by static images with implied motion. J. Cogn. Neurosci. 12, 48–55 (2000).

    Article  CAS  PubMed  Google Scholar 

  47. Saleem, K. S., Suzuki, W., Tanaka, K. & Hashikawa, T. Connections between anterior inferotemporal cortex and superior temporal sulcus regions in the macaque monkey. J. Neurosci. 20, 5083–5101 (2000). An anatomical study providing evidence for strong connections between IT and STS.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Giese, M. A. & Vaina, L. M. Pathways in the analysis of biological motion: computational model and fMRI results. Perception 30, Suppl. 116 (2001).

    Google Scholar 

  49. Giese, M. A. & Poggio, T. Neural mechanisms for the recognition of biological movements and actions. AI Memo 2002-012/CBCL Paper 219 (Massachusetts Institute of Technology, Cambridge, Massachusetts, 2002).

    Google Scholar 

  50. Perrett, D. I. & Oram, M. W. Neurophysiology of shape processing. Image Vis. Comput. 11, 317–333 (1993).

    Article  Google Scholar 

  51. Rolls, E. T. & Milward, T. A model of invariant object recognition in the visual system: learning rules, activation functions, lateral inhibition, and information-based performance measures. Neural Comput. 12, 2547–2572 (2000).

    Article  CAS  PubMed  Google Scholar 

  52. Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. (Lond.) 160, 106–154 (1962).

    Article  CAS  Google Scholar 

  53. Jones, J. P. & Palmer, L. A. An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. J. Neurophys. 58, 1233–1258 (1987). | PubMed |

    Article  CAS  Google Scholar 

  54. Dow, B. M., Snyder, A. Z., Vautin, R. G. & Bauer, R. Magnification factor and receptive field size in foveal striate cortex of the monkey. Exp. Brain Res. 44, 213–228 (1981).

    Article  CAS  PubMed  Google Scholar 

  55. Hegdé, J. & van Essen, D. C. Selectivity for complex shapes in primate visual area V2. J. Neurosci. 20, RC61 (2000).

    Article  PubMed  PubMed Central  Google Scholar 

  56. Gallant, J. L., Braun, J. & van Essen, D. C. Selectivity for polar, hyperbolic, and Cartesian gratings in macaque visual cortex. Science 259, 100–103 (1993).

    Article  CAS  PubMed  Google Scholar 

  57. Pinon, M. C., Gattass, R. & Sousa, A. P. Area V4 in Cebus monkey: extent and visuotopic organization. Cereb. Cortex 8, 685–701 (1998).

    Article  CAS  PubMed  Google Scholar 

  58. Pasupathy, A. & Connor, C. E. Response to contour features in macaque area V4. J. Neurophysiol. 82, 2490–2502 (1999).

    Article  CAS  PubMed  Google Scholar 

  59. Hubel, D. & Wiesel, T. Ferrier lecture. Functional architecture of the macaque monkey visual cortex. Proc. R. Soc. Lond. B 198, 1–59 (1977).

    Article  CAS  PubMed  Google Scholar 

  60. Fukushima, K. Neocognitron: a self-organizing neural network model for a mechanisms of pattern recognition unaffected by shift in position. Biol. Cybern. 36, 193–202 (1980). A classical neural network model for object recognition that uses a hierarchy of neural detectors in combination with pooling through a maximum operation to account for invariant recognition.

    Article  CAS  PubMed  Google Scholar 

  61. Mel, B. W. & Fieser, J. Minimizing binding errors using learned conjunctive features. Neural Comp. 12, 731–762 (2000).

    Article  CAS  Google Scholar 

  62. Lampl, I., Riesenhuber, M., Poggio, T. & Ferster, D. The max operation in cells in the cat visual cortex. Soc. Neurosci. Abstr. 30, 619 (2001).

    Google Scholar 

  63. Gawne, T. J. & Martin, J. Response of primate visual cortical V4 neurons to two simultaneously presented stimuli. J. Neurophysiol. 88, 1128–1135 (2002). An electrophysiological study showing evidence for maximum computation in some neurons in area V4 of monkeys.

    Article  PubMed  Google Scholar 

  64. Logothetis, N. K. & Sheinberg, D. L. Visual object recognition. Annu. Rev. Neurosci. 19, 577–621 (1996).

    Article  CAS  PubMed  Google Scholar 

  65. Tanaka, K. Inferotemporal cortex and object vision. Annu. Rev. Neurosci. 19, 109–139 (1996).

    Article  CAS  PubMed  Google Scholar 

  66. Young, M. & Yamane, S. Sparse population coding of faces in the inferotemporal cortex. Science 256, 1327–1331 (1992).

    Article  CAS  PubMed  Google Scholar 

  67. Rolls, E. T. & Tovee, M. J. Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. Exp. Brain Res. 73, 713–726 (1995). | PubMed |

    CAS  Google Scholar 

  68. Saito, H. in Brain Mechanisms of Perception and Memory (eds Ono, T., Squire, L. R., Raichle, M. E., Perrett, D. I. & Fukuda, M.) 121–140 (Oxford Univ. Press, Oxford, UK, 1993). A review article about the dorsal pathway indicating that the analysis of complex optical flow patterns is accomplished by a hierarchy of motion-selective areas, similar to form analysis in the ventral stream.

    Google Scholar 

  69. Smith, A. T. & Snowden, R. J. Visual Detection of Motion (Academic, London, UK, 1994).

    Google Scholar 

  70. Rodman, H. R. & Albright, T. D. Single-unit analysis of pattern-motion selective properties in the middle temporal visual area (MT). Exp. Brain Res. 75, 53–64 (1989).

    Article  CAS  PubMed  Google Scholar 

  71. Poggio, T. & Reichardt, W. Visual control of orientation behavior in the fly. Part II: towards the underlying neural interactions. Quart. Rev. Biophys. 9, 377–438 (1976).

    Article  CAS  Google Scholar 

  72. Heeger, D. J. Optical flow using spatiotemporal filters. Int. J. Comput. Vis. 1, 279–302 (1988).

    Article  Google Scholar 

  73. Grossberg, S., Mingolla, E. & Viswanathan, L. Neural dynamics of motion integration and segmentation within and across apertures. Vision. Res. 41, 2521–2553 (2001).

    Article  CAS  PubMed  Google Scholar 

  74. Sereno, M. E. Neural Computation of Pattern Motion (MIT Press, Cambridge, Massachusetts, 1993).

    Google Scholar 

  75. Nowlan, S. J. & Sejnowski, T. J. A selection model for motion processing in area MT of primates. J. Neurosci. 15, 1195–1214 (1995).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  76. Simoncelli, E. P. & Heeger, D. J. A model of neuronal responses in visual area MT. Vision. Res. 38, 743–761 (1998).

    Article  CAS  PubMed  Google Scholar 

  77. Albright, T. D. & Desimone, R. Local precision of visuotopic organization in the middle temporal area (MT) of the macaque. Exp. Brain Res. 65, 582–592 (1987).

    Article  CAS  PubMed  Google Scholar 

  78. Lagae, L., Raiguel, S. & Orban, G. A. Speed and direction selectivity of macaque middle temporal neurons. J. Neurophysiol. 69, 19–39 (1993).

    Article  CAS  PubMed  Google Scholar 

  79. Allman, J., Miezin, F., & McGuinness, E. Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception 14, 105–126 (1985).

    Article  CAS  PubMed  Google Scholar 

  80. Xiao, D. K. et al. Spatial heterogeneity of inhibitory surrounds in the middle temporal visual area. Proc. Natl Acad. Sci. USA 92, 11303–11306 (1995).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  81. Born, R. T. Center-surround interactions in the middle temporal visual area of the owl monkey. J. Neurophysiol. 84, 2658–2669 (2000).

    Article  CAS  PubMed  Google Scholar 

  82. Tanaka, K., Fukuda, Y. & Saito, H. Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. J. Neurophysiol. 62, 626–641 (1989).

    Article  CAS  PubMed  Google Scholar 

  83. Eifuku, S. & Wurtz, R. H. Response to motion in extrastiate area MSTl: center-surround interactions. J. Neurophysiol. 80, 282–296 (1998).

    Article  CAS  PubMed  Google Scholar 

  84. Orban, G. et al. A motion area in human visual cortex. Proc. Natl Acad. Sci. USA 92, 993–997 (1995).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  85. Dupont, P. et al. The kinetic occipital region in human visual cortex. Cereb. Cortex 7, 283–292 (1997). References 84 and 85 are fMRI studies describing an area in the occipital cortex — KO — that responds particularly strongly for motion discontinuities.

    Article  CAS  PubMed  Google Scholar 

  86. Graziano, M. A., Andersen, R. A. & Snowden, R. J. Tuning of MST neurons to spiral motions. J. Neurosci. 14, 54–67 (1994).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  87. Geesaman, B. J. & Andersen, R. A. The analysis of complex motion patterns by form/cue invariant MSTd neurons. J. Neurosci. 16, 4716–4732 (1996).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  88. Xie, X. & Giese, M. A. Nonlinear dynamics of direction-selective nonlinear neural media. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 65, 051904 (2002).

    Article  PubMed  CAS  Google Scholar 

  89. Markram, H., Lübke, J., Frotscher, M. & Sakmann, B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213–215 (1997). A study showing that synaptic changes depend on the precise timing of pre- and postsynaptic activity. Spike-time-dependent plasticity is a biologically plausible mechanism for learning temporal order.

    Article  CAS  PubMed  Google Scholar 

  90. Sejnowski, T. J. The book of Hebb. Neuron 24, 773–776 (1999).

    Article  CAS  PubMed  Google Scholar 

  91. Giese, M. A., Jastorff, J. & Kourtzi, Z. Learning of the discrimination of artificial complex biological motion. Perception 31, Suppl. 117 (2002).

    Google Scholar 

  92. Mineiro, P. & Zipser, D. Analysis of direction selectivity arising from recurrent cortical interactions. Neural Comput. 10, 353–371 (1998).

    Article  CAS  PubMed  Google Scholar 

  93. Miyashita, Y. & Chang, H. S. Neural correlate of pictorial short-term memory in the primate temporal cortex. Nature 331, 68–70 (1988).

    Article  CAS  PubMed  Google Scholar 

  94. Stryker, M. P. Neurobiology. Temporal associations. Nature 354, 108–109 (1991).

    Article  CAS  PubMed  Google Scholar 

  95. Koch, C., Poggio, T. & Torre, V. Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing. Proc. Natl Acad. Sci. USA 80, 2799–2802 (1983).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  96. Dittrich, W. H. Action categories and the perception of biological motion. Perception 22, 15–22 (1993).

    Article  CAS  PubMed  Google Scholar 

  97. Kozlowski, L. T. & Cutting, J. E. Recognizing the sex of a walker from a dynamic point light display. Percept. Psychophys. 21, 575–580 (1977). A classical study showing that subjects can recognize gender from point-light walkers.

    Article  Google Scholar 

  98. Cutting, J. E. & Kozlowski, L. T. Recognizing friends by their walk: gait perception without familiarity cues. Bull. Psychon. Soc. 9, 353–356 (1977). A classical study showing that subjects can recognize identity from point-light walkers.

    Article  Google Scholar 

  99. Dittrich, W. H., Troscianko, T., Lea, S. E. & Morgan, D. Perception of emotion from dynamic point-light displays represented in dance. Perception 25, 727–738 (1996).

    Article  CAS  PubMed  Google Scholar 

  100. Pollick, F. E., Paterson, H. M., Bruderlin, A. & Sanford, A. J. Perceiving affect from arm movement. Cognition 82, B51–B61 (2001).

    Article  CAS  PubMed  Google Scholar 

  101. Giese, M. A. & Lappe, M. Measuring generalization fields for the recognition of biological motion. Vision Res. 42, 1847–1858 (2002). A psychophysical study of the categorization of biological motion using linear spaces of movement patterns defined by motion morphing.

    Article  CAS  PubMed  Google Scholar 

  102. Troje, N. Decomposing biological motion: A framework for analysis and synthesis of human gait patterns. J. Vis. 2, 371–387 (2002).

    Article  PubMed  Google Scholar 

  103. Hill, H. & Pollick, F. E. Exaggerating temporal differences enhances recognition of individuals from point light displays. Psychol. Sci. 11, 223–228 (2000). A psychophysical study showing that, by motion morphing, temporal caricatures of arm movement patterns can be generated that improve the recognition of identity from point-light walkers.

    Article  CAS  PubMed  Google Scholar 

  104. Giese, M. A. & Poggio, T. Morphable models for the analysis and synthesis of complex motion patterns. Int. J. Comp. Vis. 38, 59–73 (2000).

    Article  Google Scholar 

  105. Verfaillie, K., de Troy, A. & van Rensbergen, J. Transsaccadic integration of biological motion. J. Exp. Psychol. Learn. Mem. Cogn. 20, 649–670 (1994).

    Article  CAS  PubMed  Google Scholar 

  106. Pavlova, M. Biological motion perception under various presentation rates. Perception 24, Suppl. 112 (1995).

    Google Scholar 

  107. Sumi, S. Upside-down presentation of the Johansson moving light-spot pattern. Perception 13, 283–302 (1984). A psychophysical experiment showing impaired biological motion recognition for upside-down stimuli.

    Article  CAS  PubMed  Google Scholar 

  108. Pavlova, M. & Sokolov, S. Orientation specificity in biological motion perception. Percept. Psychophys. 62, 889–899 (2000).

    Article  CAS  PubMed  Google Scholar 

  109. Bülthoff, I., Bülthoff, H. H. & Sinha, P. Top–down influences on stereoscopic depth-perception. Nature Neurosci. 1, 254–257 (1998). A psychophysical study showing that biological motion recognition is view dependent, and that the depth information indicated by the structure of the walker dominates over the stereoscopically specified depth of the walker dots.

    Article  PubMed  Google Scholar 

  110. Grossman, E. D. & Blake, R. Brain activity evoked by inverted and imagined biological motion. Vision Res. 41, 1475–1482 (2001).

    Article  CAS  PubMed  Google Scholar 

  111. Marr, D. & Nishihara, H. K. Representation and recognition of the spatial organization of three-dimensional shapes. Proc. R. Soc. Lond. B 200, 269–294 (1978). A paper on the recognition of articulated figures in terms of structural three-dimensional models.

    Article  CAS  PubMed  Google Scholar 

  112. Hogg, D. Model-based vision: a program to see a walking person. Image Vis. Comp. 1, 5–19 (1983).

    Article  Google Scholar 

  113. Hoffman, D. D. & Flinchbaugh, B. E. The interpretation of biological motion. Biol. Cybern. 42, 195–204 (1982).

    Article  CAS  PubMed  Google Scholar 

  114. Webb, J. A. & Aggarwal, J. K. Structure from motion of rigid and jointed objects. Artif. Intell. 19, 107–130 (1982). A classical paper on the estimation of the structure of articulated objects based on geometrical constraints.

    Article  Google Scholar 

  115. Oram, M. W. & Perrett, D. I. Neural processing of biological motion in the macaque temporal cortex. In Computational Vision Based on Neurobiology. (ed. Lawton, T. B.) SPIE Proc. 2054, 155–165 (1994).

    Chapter  Google Scholar 

  116. Pinto, J. & Shiffrar, M. Subconfigurations of the human form in the perception of biological motion displays. Acta Psychol. 102, 293–318 (1999).

    Article  CAS  Google Scholar 

  117. Cutting, J. E., Moore, C. & Morrison, R. Masking the motions of human gait. Percept. Psychophys. 44, 339–347 (1988). A psychophysical experiment showing that point-light walkers can be recognized even in the presence of a substantial number of moving background dots and scrambled walker noise.

    Article  CAS  PubMed  Google Scholar 

  118. Bertenthal, B. I. J. & Pinto, J. Global processing of biological motion. Psychol. Sci. 5, 221–225 (1994).

    Article  Google Scholar 

  119. Thornton, I. M., Rensink, R. A. & Shiffrar, M. Active versus passive processing of biological motion. Perception 31, 837–853 (2002). A psychophysical study using a dual-task experiment to show that the tolerance of biological motion recognition against masking depends on attention.

    Article  PubMed  Google Scholar 

  120. McLeod, P., Dittrich, W., Driver, J., Perrett, D. & Zihl, J. Preserved and impaired detection of structure from motion by a 'motion-blind' patient. Vis. Cogn. 3, 363–391 (1996).

    Article  Google Scholar 

  121. Vaina, L. M., Lemay, M., Bienfang, D. C., Choi, A. Y. & Nakayama, K. Intact 'biological motion' and 'structure from motion' perception in a patient with impaired motion mechanisms: a case study. Vis. Neurosci. 5, 353–369 (1990). References 120 and 121 describe studies of patients that have lesions in the dorsal pathway and that can still recognize biological motion.

    Article  CAS  PubMed  Google Scholar 

  122. Neri, P., Morrone, C. & Burr, D. C. Seeing biological motion. Nature 395, 894–896 (1998).

    Article  CAS  PubMed  Google Scholar 

  123. Logothetis, N. K., Pauls, J., Augath, M., Trinath, T. & Oeltermann, A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157 (2001).

    Article  CAS  PubMed  Google Scholar 

  124. Grill-Spector, K. & Malach, R. fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol. 107, 293–321 (2001).

    Article  CAS  Google Scholar 

  125. Poggio, T. A theory of how the brain might work. Proc. Cold Spring Harb. Symp. Quant. Biol. 55, 899–910 (1990).

    Article  CAS  Google Scholar 

  126. Wallis, G. & Rolls, E. T. Invariant face and object recognition in the visual system. Prog. Neurobiol. 51, 167 (1997).

    Article  CAS  PubMed  Google Scholar 

  127. Fox, R. & McDaniel, C. The perception of biological motion by human infants. Science 218, 486–487 (1982).

    Article  CAS  PubMed  Google Scholar 

  128. Cavanagh, P., Labianca, A. P. & Thornton, I. M. Attention-based visual routines: sprites. Cognition 80, 47–60 (2001). A psychophysical study using a visual search experiment to show that biological motion recognition in more complex tasks requires attention.

    Article  CAS  PubMed  Google Scholar 

  129. Shiffrar, M. & Freyd, J. J. Apparent motion of the human body. Psychol. Sci. 1, 257–264 (1993). A psychophysical investigation of the perceived paths of apparent motion between two photographs of human body configurations. For small ISIs, motion along the shortest paths is perceived, so the percept is dominated by low-level motion perception. For long ISIs, subjects perceive the biologically possible paths, indicating top–down influences from motion recognition.

    Article  Google Scholar 

  130. Umiltà, M. A. et al. I know what you are doing: a neurophysiological study. Neuron 31, 155–165 (2001).

    Article  PubMed  Google Scholar 

  131. Verfaille, K. & Daems, A. Representing and anticipating actions in vision. Vis. Cogn. 9, 217–232 (2002).

    Article  Google Scholar 

  132. Bassili, J. N. Facial motion in the perception of faces and of emotional expression. J. Exp. Psychol. Hum. Percept. Perform. 4, 373–379 (1978).

    Article  CAS  PubMed  Google Scholar 

  133. Poizner, H., Bellugi, U. & Lutes-Driscoll, V. Perception of American Sign Language in dynamic point-light display. J. Exp. Psychol. Hum. Percept. Perform. 7, 430–440 (1981).

    Article  CAS  PubMed  Google Scholar 

  134. Runeson, S. & Frykholm, G. Visual perception of lifted weight. J. Exp. Psychol. Hum. Percept. Perform. 7, 733–740 (1981).

    Article  CAS  PubMed  Google Scholar 

  135. Verfaillie, K. Perceiving human locomotion: priming effects in direction discrimination. Brain Cogn. 44, 192–213 (2000). A sound review of the experimental studies on biological motion perception that argues in favour of a view-based representation of biological motion patterns.

    Article  CAS  PubMed  Google Scholar 

  136. Goddard, N. H. The Perception of Articulated Motion: Recognizing Moving Light Displays. Thesis, Dept. of Computer Science, Univ. of Rochester (1992). A connectionist model for biological motion recognition that proposes a similar architecture to the model used in this review.

    Google Scholar 

  137. Little, J. J. & Boyd, J. Recognizing people by their gait: the shape of motion. J. Videre 1, 1–32 (1998).

    Google Scholar 

  138. Rosenblum, M., Yacoob, Y. & Davis, L. S. Human expression recognition from motion using a radial basis function network architecture. IEEE Trans. Neural Networks 7, 1121–1138 (1996).

    Article  CAS  PubMed  Google Scholar 

  139. Cedras, C. & Shah, M. Motion-based recognition: a survey. Image Vis. Comp. 13, 129–155 (1995).

    Article  Google Scholar 

  140. Gavrila, D. M. The visual analysis of human movement: a survey. Comp. Vis. Image Underst. 73, 82–98 (1999).

    Article  Google Scholar 

  141. Aggarwal, J. K. & Cai, Q. Human motion analysis: a review. Comp. Vis. Image Underst. 73, 428–440 (1999). References 139–141 review computer vision methods for the recognition of biological movements.

    Article  Google Scholar 

  142. Song, Y., Goncalves, L., di Bernardo, E. & Perona, P. Monocular perception of biological motion in Johansson displays. Comp. Vis. Image Underst. 81, 303–327 (2001).

    Article  Google Scholar 

Download references

Acknowledgements

We thank R. Blake, D. Perrett, T. Sejnowski, C. Koch, S. Edelman, I. Bülthoff, E. Curio, Z. Kourtzi, M. Riesenhuber, P. Sinha, I. Thornton and L. Vaina for comments; A. Benali, Z. Kourtzi and C. Curio for help with data acquisition; and the Max-Planck Institute for Biological Cybernetics, Tübingen, for providing support. M.G. was supported by the Deutsche Forschungsgemeinschaft, Honda R&D Americas Inc. and the Deusche Volkswagen Stiftung. T.P. is supported in part by the Eugene McDermott chair. Research at CBCL was sponsored by the National Science Foundation. Additional support is provided by Central Research Institute of Electric Power Industry, Eastman Kodak Company, DaimlerChrysler AG, Compaq, Komatsu Ltd., NEC Fund, Nippon Telegraph & Telephone, Siemens Corporate Research Inc., Honda and Toyota.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Martin A. Giese.

Supplementary information

Supplementary information: Movies that illustrate the stimuli that have been used in experiments to study the recognition of biological movements.

Movie 1 (MOV 626 kb)

Full-body stimulus walking

Movie 2

Full-body stimulus running (MOV 100 kb)

Movie 3

Motion morph between walking and running1011 = α2 = 0.5). (MOV 626 kb)

Movie 4

Point-light stimulus walking6. (MOV 626 kb)

Movie 5

Point-light stimulus running6. (MOV 626 kb)

Movie 6

Point-light stimulus rotated by 90° (Refs 107, 108). (MOV 626 kb)

Movie 7

Scrambled point-light walker117. (MOV 626 kb)

Movie 8

Point-light walker, hands and feet missing27,116. (MOV 626 kb)

Movie 9

Point-light walker with strongly degraded local motion information30. (MOV 642 kb)

Related links

Related links

FURTHER INFORMATION

ARL homepage

Biological Motion Demonstrations

George Mather

Randolph Blake

BioMotionLab

CBCL homepage

MIT Encyclopedia of Cognitive Sciences

computational vision

object recognition, animal studies

structure from visual information sources

vision and learning

Glossary

BIOLOGICAL MOVEMENT

In psychophysics, 'biological motion' often refers specifically to point-light stimuli. We use the term 'biological movements' to characterize stimuli that show the movements of animals and humans, independent of the presentation mode.

SUPERIOR TEMPORAL SULCUS

(STS). A sulcus in the temporal lobe of monkey and human cortex that contains areas that are selectively activated by biological movements.

MULTIMODAL

Neurons that respond to sensory input in more than one modality, for example, both visual and auditory stimuli.

OPTIC FLOW

A field of motion vectors that specifies how points of a frame (in an image sequence) are displaced over time. Unlike densely textured scenes, point-light stimuli do not specify a dense, spatially continuous, optic-flow field.

NEURAL FEATURE DETECTORS

Neurons in cortex can often be interpreted as graded 'detectors' that are activated when specific features (such as orientation, corners, local motion with defined speed and direction, and faces) are present in their receptive fields.

GABOR-LIKE FILTERS

Gabor functions are defined by sinusoidal functions that are windowed by a gaussian function. They define filters that are localized in the spatial domain as well as in the spatial frequency domain.

MAXIMUM-LIKE OPERATION

An operation that results in an output signal that approximates the maximum among several input signals. Maximum computation can be approximated by physiologically plausible neural circuits.

GAUSSIAN RADIAL BASIS FUNCTIONS

(RBF). Units used to model neurons that are tuned for complex stimuli. Their activation is described by a multi-dimensional gaussian function that depends on the difference between input signals and a constant vector (RBF centre) that defines the input that induces maximal activity.

LEAKY INTEGRATOR

A simplified model for the dynamics of the membrane potential of a neuron. The dynamics are given by a linear differential equation.

REICHARDT DETECTOR

A simple model for local motion detectors, originally studied in the eye of the beetle and the fly. The Reichardt detector multiplies the output signals of two receptors with different positions in the visual field, after delaying or low-pass filtering the output of one of them. The output of the detector is direction-selective. Versions of the Reichardt detector have been used to model direction-selective neurons in the primary visual cortex.

MOTION MORPHING

An algorithm that continuously interpolates between different movement patterns, for example, walking and running. Good morphing algorithms result in interpolations that are very similar to natural movements.

OCTAVE

The interval between any two frequencies that have a ratio of 2 to 1.

HIDDEN MARKOV MODEL

A finite set of states, each of which is associated with a probability distribution. Transitions among the states are governed by a set of so-called transition probabilities. In a given state, an outcome can be generated according to the associated probability distribution. Only the outcome, not the state, is visible to an external observer and so the states are 'hidden'.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Giese, M., Poggio, T. Neural mechanisms for the recognition of biological movements. Nat Rev Neurosci 4, 179–192 (2003). https://doi.org/10.1038/nrn1057

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrn1057

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing