Humans can recognize biological movements, such as walking, accurately and robustly. This review uses a neurophysiologically plausible and quantitative model as a tool for organizing and making sense of the available experimental data, despite its growing size and complexity.
Most experimental results can be accounted for by simple neural mechanisms, under the two key assumptions that recognition is based on a hierarchical feedforward cortical architecture and learned prototypical patterns. Such prototypes might be stored in specific neurons in the visual system.
The model shows that recognition of biological movements can be achieved with plausible neural mechanisms in a way that is quantitatively consistent with the experimental data on pattern selectivity, view dependence and robustness of recognition.
The model comprises two parallel pathways, one corresponding to the dorsal pathway (specialized for the analysis of motion information) and one to the ventral pathway (specialized for the analysis of form information). In each pathway, neural feature detectors extract form or optic-flow features with increasing complexity along the hierarchy. The position and size invariance of the feature detectors also increases along the hierarchy. Experimental data and quantitative simulations indicate that the ventral and dorsal pathways are both needed for the recognition of normal biological movement stimuli, whereas the recognition of point-light stimuli seems mainly to depend on the dorsal pathway.
The proposed architecture predicts the existence of neurons that can learn to respond selectively to new biological movement patterns. It also predicts that arbitrary complex movement patterns should be learnable, as long as they provide suitable stimulation of the mid- and low-level feature detectors of the two pathways.
The model predicts the existence of neurons in the dorsal pathway that become selectively activated by complex optic-flow patterns that arise for biological movement patterns.
It is demonstrated that attention and top–down influences are not required to solve the basic tasks of motion recognition. These factors may be necessary for more sophisticated motion recognition tasks. The model cannot account for such influences of attention and of different tasks.
A number of open questions and predictions of the model are considered. The use of a quantitative model allows us to generate specific predictions and to show that a neurophysiologically consistent, learning-based, feedforward model can reproduce many key experimental results. Open questions include how information from the two pathways is integrated, and which neural mechanisms underlie sequence selectivity in both pathways.
The visual recognition of complex movements and actions is crucial for the survival of many species. It is important not only for communication and recognition at a distance, but also for the learning of complex motor actions by imitation. Movement recognition has been studied in psychophysical, neurophysiological and imaging experiments, and several cortical areas involved in it have been identified. We use a neurophysiologically plausible and quantitative model as a tool for organizing and making sense of the experimental data, despite their growing size and complexity. We review the main experimental findings and discuss possible neural mechanisms, and show that a learning-based, feedforward model provides a neurophysiologically plausible and consistent summary of many key experimental results.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
Scientific Reports Open Access 28 June 2023
Scientific Reports Open Access 19 May 2023
Nature Human Behaviour Open Access 16 January 2023
Subscribe to this journal
Receive 12 print issues and online access
$189.00 per year
only $15.75 per issue
Rent or buy this article
Prices vary by article type
Prices may be subject to local taxes which are calculated during checkout
Ewert, J. -P. Neuroethology of releasing mechanisms: Prey-catching in toads. Behav. Brain Sci. 10, 337–405 (1987).
Morris, D. The reproductive behaviour of the zebra finch (Poephilla guttata), with special reference to pseudofemale behaviour. Behaviour 6, 271–322 (1954).
Darwin, C. The Expressions of the Emotions in Man and Animals (Appleton, New York, 1872).
Andrew, R. J. Evolution of facial expression. Science 142, 1034–1041 (1963).
Adolphs, R. The neurobiology of social cognition. Curr. Opin. Neurobiol. 11, 231–239 (2001).
Johansson, G. Visual perception of biological motion and a model for its analysis. Percept. Psychophys. 14, 201–211 (1973). A classical paper reporting the original experiment in which subjects were able to recognize complex actions from point-light displays that were generated by filming moving actors.
Bruce, C., Desimone, R. & Gross, C. G. Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J. Neurophysiol. 46, 369–384 (1981).
Perrett, D. I. et al. Visual analysis of body movements by neurones in the temporal cortex in the macaque monkey: a preliminary report. Behav. Brain Res. 16, 153–170 (1985).
Perrett, D. I. et al. in AI and the Eye (eds Blake, A. & Troscianko, T.) 181–200 (John Wiley, Chichester, 1990).
Oram, M. W. & Perrett, D. I. Responses of anterior superior temporal polysensory (STPa) neurons to 'biological motion' stimuli. J. Cogn. Neurosci. 6, 99–116 (1994). References 8–10 present electrophysiological studies in macaque monkeys with recordings in different parts of the STS during presentation of biological movement stimuli. In different areas, neurons were found that seemed to respond selectively for biological movements of normal and point-light stimuli.
Jellema, T., Baker, C. I., Wicker, B. & Perrett, D. I. Neural representation for the perception of the intentionality of actions. Brain Cogn. 44, 280–302 (2000).
Perrett, D. I. et al. Frameworks of analysis for the neural representation of animate objects and actions. Neural representation for the perception of the intentionality of actions. J. Exp. Biol. 146, 87–113 (1989).
Rizzolatti, G., Fogassi, L. & Gallese, V. Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Rev. Neurosci. 2, 661–670 (2001). A review article about the neural basis of action recognition and imitation with a special focus on the 'mirror neuron' system.
Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. Action recognition in the premotor cortex. Brain 119, 593–609 (1996).
Rizzolatti, G. & Arbib, M. A. Language within our grasp. Trends Neurosci. 21, 188–194 (1998).
Kohler, E et al. Hearing sounds, understanding actions: action representation in mirror neurons. Science 297, 846–848 (2002).
Decety, J. & Grèzes, J. Neural mechanisms subserving the perception of human actions. Trends Cogn. Sci. 3, 172–178 (1999). A review article about functional imaging studies of the perception of human actions.
Allison, T., Puce, A. & McCarthy, G. Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267–278 (2000).
Grossman, E. D. & Blake, R. Brain areas active during visual perception of biological motion. Neuron 35, 1167–1175 (2002). An fMRI study that compares activity in different form- and motion-selective areas for biological and non-biological motion.
Bonda, E. et al. Specific involvement of human parietal systems and the amygdala in the perception of biological motion J. Neurosci. 16, 3737–3744 (1996). One of the first fMRI studies showing activity that is specific for biological motion in the human STS.
Howard, R. J. et al. A direct demonstration of functional specialization within motion-related visual and auditory cortex of the human brain. Curr. Biol. 6, 1015–1019 (1996).
Grossman, E, et al. Brain areas involved in perception of biological motion. J. Cogn. Neurosci. 12, 711–720 (2000).
Vaina, L. M. et al. Functional neuroanatomy of biological motion perception in humans. Proc. Natl Acad. Sci USA 98, 11656–11661 (2001). An fMRI study that shows an involvement of the opponent motion-selective area KO in biological motion recognition, and an influence of the task on the identified biological motion-sensitive areas.
Iacoboni, M. et al. Cortical mechanisms of human imitation. Science 286, 2526–2528 (1999).
Buccino, G. et al. Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur. J. Neurosci. 13, 400–404 (2001).
Servos, P., Osu, R., Santi, A. & Kawato, M. The neural substrates of biological motion perception: an fMRI study. Cereb. Cortex 12, 772–782 (2002).
Mather, G., Radford, K. & West, S. Low-level visual processing of biological motion. Proc. R. Soc. Lond. B 249, 149–155 (1992).
Bobick, A. F. Movement, activity, and action: the role of knowledge in the perception of motion. Phil. Trans. R. Soc. Lond. B 352, 1257–1265 (1997).
Todd, J. T. Perception of gait. J. Exp. Psychol. Hum. Percept. Perform. 9, 31–42 (1993).
Beintema, J. A. & Lappe, M. Perception of biological motion without local image motion. Poc. Natl Acad. Sci. USA 99, 5661–5663 (2002).
Downing, P., Jiang, Y., Shuman, M. & Kanwisher, N. A cortical area selective for visual processing of the human body. Science 293, 2470–2473 (2001).
Pollick, F. E., Fidopiastis, C. & Braden, V. Recognizing the style of spatially exaggerated tennis serves. Perception 30, 323–338 (2001). A psychophysical experiment using morphing to generate exaggerations of biological movements. The recognition of spatially exaggerated patterns is improved.
Hill, H. & Johnston, A. Categorizing sex and identity from the biological motion of faces. Curr. Biol. 11, 880–885 (2001). A psychophysical study showing that identity can be recognized from facial movements even when identity-specific form information is removed from the face.
Tarr, M. J. & Bülthoff, H. H. Image-based object recognition in man, monkey and machine. Cognition 67, 1–20 (1998). A review article about view-based object recognition in biological and technical systems.
Poggio, T. & Edelman, S. A network that learns to recognize three-dimensional objects. Nature 343, 263–266 (1990).
Logothetis, N. K., Pauls, J. & Poggio, T. Shape representation in the inferior temporal cortex of monkeys. Curr. Biol. 5, 552–563 (1995). An electrophysiological study showing that neurons in area IT of monkey cortex can learn to become selective for artificial three-dimensional shapes. This study also provides evidence for the neural encoding of complex three-dimensional shapes in terms of prototypical views and in particular for size and position invariance being independent of specific experience.
Ungerleider, L. G. & Mishkin, M. in Analysis of Visual Behavior (eds Ingle, D. J., Goodale, M. A. & Mansfield, R. J. W.) 549–586 (MIT Press, Cambridge, Massachusetts, 1982).
Felleman, D. J. & van Essen, D. C. Distributed hierarchical processing in the primate visual cortex. Cereb. Cortex 1, 1–47 (1991).
Milner, A. D. & Goodale, M. A. Visual pathways to perception and action. Prog. Brain Res. 95, 317–337 (1993).
Thorpe, S., Fize, D. & Marlot, C. Speed of processing in the human visual system. Nature 381, 520–522 (1996). An electroencephalography study showing that selective cortical activity in a categorization task can be detected as early as 150 ms after stimulus onset. This makes it unlikely that complex feedback circuits are needed for basic, 'immediate' categorization tasks.
Oram, M. W. & Perrett, D. I. Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. J. Neurophysiol. 76, 109–129 (1996).
Johansson, G. Spatio-temporal differentiation and integration in visual motion perception. Psychol. Res. 38, 379–393 (1976).
Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition. Nature Neurosci. 2, 1019–1025 (1999). A neurophysiologically plausible model for the recognition of stationary objects that provides the skeleton of the form pathway of our model for motion recognition. The study shows that pooling by a maximum-like operation provides feature selectivity simultaneously with sufficient position and size invariance.
Riesenhuber, M. & Poggio, T. Models of object recognition. Nature Neurosci. 3, Suppl. 1199–1204 (2000).
Riesenhuber, M. & Poggio, T. Neural mechanisms of object recognition. Curr. Opin. Neurobiol. 12, 162–168 (2002).
Kourtzi, Z. & Kanwisher, N. Activation in human MT/MST by static images with implied motion. J. Cogn. Neurosci. 12, 48–55 (2000).
Saleem, K. S., Suzuki, W., Tanaka, K. & Hashikawa, T. Connections between anterior inferotemporal cortex and superior temporal sulcus regions in the macaque monkey. J. Neurosci. 20, 5083–5101 (2000). An anatomical study providing evidence for strong connections between IT and STS.
Giese, M. A. & Vaina, L. M. Pathways in the analysis of biological motion: computational model and fMRI results. Perception 30, Suppl. 116 (2001).
Giese, M. A. & Poggio, T. Neural mechanisms for the recognition of biological movements and actions. AI Memo 2002-012/CBCL Paper 219 (Massachusetts Institute of Technology, Cambridge, Massachusetts, 2002).
Perrett, D. I. & Oram, M. W. Neurophysiology of shape processing. Image Vis. Comput. 11, 317–333 (1993).
Rolls, E. T. & Milward, T. A model of invariant object recognition in the visual system: learning rules, activation functions, lateral inhibition, and information-based performance measures. Neural Comput. 12, 2547–2572 (2000).
Hubel, D. H. & Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. (Lond.) 160, 106–154 (1962).
Jones, J. P. & Palmer, L. A. An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. J. Neurophys. 58, 1233–1258 (1987). | PubMed |
Dow, B. M., Snyder, A. Z., Vautin, R. G. & Bauer, R. Magnification factor and receptive field size in foveal striate cortex of the monkey. Exp. Brain Res. 44, 213–228 (1981).
Hegdé, J. & van Essen, D. C. Selectivity for complex shapes in primate visual area V2. J. Neurosci. 20, RC61 (2000).
Gallant, J. L., Braun, J. & van Essen, D. C. Selectivity for polar, hyperbolic, and Cartesian gratings in macaque visual cortex. Science 259, 100–103 (1993).
Pinon, M. C., Gattass, R. & Sousa, A. P. Area V4 in Cebus monkey: extent and visuotopic organization. Cereb. Cortex 8, 685–701 (1998).
Pasupathy, A. & Connor, C. E. Response to contour features in macaque area V4. J. Neurophysiol. 82, 2490–2502 (1999).
Hubel, D. & Wiesel, T. Ferrier lecture. Functional architecture of the macaque monkey visual cortex. Proc. R. Soc. Lond. B 198, 1–59 (1977).
Fukushima, K. Neocognitron: a self-organizing neural network model for a mechanisms of pattern recognition unaffected by shift in position. Biol. Cybern. 36, 193–202 (1980). A classical neural network model for object recognition that uses a hierarchy of neural detectors in combination with pooling through a maximum operation to account for invariant recognition.
Mel, B. W. & Fieser, J. Minimizing binding errors using learned conjunctive features. Neural Comp. 12, 731–762 (2000).
Lampl, I., Riesenhuber, M., Poggio, T. & Ferster, D. The max operation in cells in the cat visual cortex. Soc. Neurosci. Abstr. 30, 619 (2001).
Gawne, T. J. & Martin, J. Response of primate visual cortical V4 neurons to two simultaneously presented stimuli. J. Neurophysiol. 88, 1128–1135 (2002). An electrophysiological study showing evidence for maximum computation in some neurons in area V4 of monkeys.
Logothetis, N. K. & Sheinberg, D. L. Visual object recognition. Annu. Rev. Neurosci. 19, 577–621 (1996).
Tanaka, K. Inferotemporal cortex and object vision. Annu. Rev. Neurosci. 19, 109–139 (1996).
Young, M. & Yamane, S. Sparse population coding of faces in the inferotemporal cortex. Science 256, 1327–1331 (1992).
Rolls, E. T. & Tovee, M. J. Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. Exp. Brain Res. 73, 713–726 (1995). | PubMed |
Saito, H. in Brain Mechanisms of Perception and Memory (eds Ono, T., Squire, L. R., Raichle, M. E., Perrett, D. I. & Fukuda, M.) 121–140 (Oxford Univ. Press, Oxford, UK, 1993). A review article about the dorsal pathway indicating that the analysis of complex optical flow patterns is accomplished by a hierarchy of motion-selective areas, similar to form analysis in the ventral stream.
Smith, A. T. & Snowden, R. J. Visual Detection of Motion (Academic, London, UK, 1994).
Rodman, H. R. & Albright, T. D. Single-unit analysis of pattern-motion selective properties in the middle temporal visual area (MT). Exp. Brain Res. 75, 53–64 (1989).
Poggio, T. & Reichardt, W. Visual control of orientation behavior in the fly. Part II: towards the underlying neural interactions. Quart. Rev. Biophys. 9, 377–438 (1976).
Heeger, D. J. Optical flow using spatiotemporal filters. Int. J. Comput. Vis. 1, 279–302 (1988).
Grossberg, S., Mingolla, E. & Viswanathan, L. Neural dynamics of motion integration and segmentation within and across apertures. Vision. Res. 41, 2521–2553 (2001).
Sereno, M. E. Neural Computation of Pattern Motion (MIT Press, Cambridge, Massachusetts, 1993).
Nowlan, S. J. & Sejnowski, T. J. A selection model for motion processing in area MT of primates. J. Neurosci. 15, 1195–1214 (1995).
Simoncelli, E. P. & Heeger, D. J. A model of neuronal responses in visual area MT. Vision. Res. 38, 743–761 (1998).
Albright, T. D. & Desimone, R. Local precision of visuotopic organization in the middle temporal area (MT) of the macaque. Exp. Brain Res. 65, 582–592 (1987).
Lagae, L., Raiguel, S. & Orban, G. A. Speed and direction selectivity of macaque middle temporal neurons. J. Neurophysiol. 69, 19–39 (1993).
Allman, J., Miezin, F., & McGuinness, E. Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception 14, 105–126 (1985).
Xiao, D. K. et al. Spatial heterogeneity of inhibitory surrounds in the middle temporal visual area. Proc. Natl Acad. Sci. USA 92, 11303–11306 (1995).
Born, R. T. Center-surround interactions in the middle temporal visual area of the owl monkey. J. Neurophysiol. 84, 2658–2669 (2000).
Tanaka, K., Fukuda, Y. & Saito, H. Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. J. Neurophysiol. 62, 626–641 (1989).
Eifuku, S. & Wurtz, R. H. Response to motion in extrastiate area MSTl: center-surround interactions. J. Neurophysiol. 80, 282–296 (1998).
Orban, G. et al. A motion area in human visual cortex. Proc. Natl Acad. Sci. USA 92, 993–997 (1995).
Dupont, P. et al. The kinetic occipital region in human visual cortex. Cereb. Cortex 7, 283–292 (1997). References 84 and 85 are fMRI studies describing an area in the occipital cortex — KO — that responds particularly strongly for motion discontinuities.
Graziano, M. A., Andersen, R. A. & Snowden, R. J. Tuning of MST neurons to spiral motions. J. Neurosci. 14, 54–67 (1994).
Geesaman, B. J. & Andersen, R. A. The analysis of complex motion patterns by form/cue invariant MSTd neurons. J. Neurosci. 16, 4716–4732 (1996).
Xie, X. & Giese, M. A. Nonlinear dynamics of direction-selective nonlinear neural media. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 65, 051904 (2002).
Markram, H., Lübke, J., Frotscher, M. & Sakmann, B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213–215 (1997). A study showing that synaptic changes depend on the precise timing of pre- and postsynaptic activity. Spike-time-dependent plasticity is a biologically plausible mechanism for learning temporal order.
Sejnowski, T. J. The book of Hebb. Neuron 24, 773–776 (1999).
Giese, M. A., Jastorff, J. & Kourtzi, Z. Learning of the discrimination of artificial complex biological motion. Perception 31, Suppl. 117 (2002).
Mineiro, P. & Zipser, D. Analysis of direction selectivity arising from recurrent cortical interactions. Neural Comput. 10, 353–371 (1998).
Miyashita, Y. & Chang, H. S. Neural correlate of pictorial short-term memory in the primate temporal cortex. Nature 331, 68–70 (1988).
Stryker, M. P. Neurobiology. Temporal associations. Nature 354, 108–109 (1991).
Koch, C., Poggio, T. & Torre, V. Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing. Proc. Natl Acad. Sci. USA 80, 2799–2802 (1983).
Dittrich, W. H. Action categories and the perception of biological motion. Perception 22, 15–22 (1993).
Kozlowski, L. T. & Cutting, J. E. Recognizing the sex of a walker from a dynamic point light display. Percept. Psychophys. 21, 575–580 (1977). A classical study showing that subjects can recognize gender from point-light walkers.
Cutting, J. E. & Kozlowski, L. T. Recognizing friends by their walk: gait perception without familiarity cues. Bull. Psychon. Soc. 9, 353–356 (1977). A classical study showing that subjects can recognize identity from point-light walkers.
Dittrich, W. H., Troscianko, T., Lea, S. E. & Morgan, D. Perception of emotion from dynamic point-light displays represented in dance. Perception 25, 727–738 (1996).
Pollick, F. E., Paterson, H. M., Bruderlin, A. & Sanford, A. J. Perceiving affect from arm movement. Cognition 82, B51–B61 (2001).
Giese, M. A. & Lappe, M. Measuring generalization fields for the recognition of biological motion. Vision Res. 42, 1847–1858 (2002). A psychophysical study of the categorization of biological motion using linear spaces of movement patterns defined by motion morphing.
Troje, N. Decomposing biological motion: A framework for analysis and synthesis of human gait patterns. J. Vis. 2, 371–387 (2002).
Hill, H. & Pollick, F. E. Exaggerating temporal differences enhances recognition of individuals from point light displays. Psychol. Sci. 11, 223–228 (2000). A psychophysical study showing that, by motion morphing, temporal caricatures of arm movement patterns can be generated that improve the recognition of identity from point-light walkers.
Giese, M. A. & Poggio, T. Morphable models for the analysis and synthesis of complex motion patterns. Int. J. Comp. Vis. 38, 59–73 (2000).
Verfaillie, K., de Troy, A. & van Rensbergen, J. Transsaccadic integration of biological motion. J. Exp. Psychol. Learn. Mem. Cogn. 20, 649–670 (1994).
Pavlova, M. Biological motion perception under various presentation rates. Perception 24, Suppl. 112 (1995).
Sumi, S. Upside-down presentation of the Johansson moving light-spot pattern. Perception 13, 283–302 (1984). A psychophysical experiment showing impaired biological motion recognition for upside-down stimuli.
Pavlova, M. & Sokolov, S. Orientation specificity in biological motion perception. Percept. Psychophys. 62, 889–899 (2000).
Bülthoff, I., Bülthoff, H. H. & Sinha, P. Top–down influences on stereoscopic depth-perception. Nature Neurosci. 1, 254–257 (1998). A psychophysical study showing that biological motion recognition is view dependent, and that the depth information indicated by the structure of the walker dominates over the stereoscopically specified depth of the walker dots.
Grossman, E. D. & Blake, R. Brain activity evoked by inverted and imagined biological motion. Vision Res. 41, 1475–1482 (2001).
Marr, D. & Nishihara, H. K. Representation and recognition of the spatial organization of three-dimensional shapes. Proc. R. Soc. Lond. B 200, 269–294 (1978). A paper on the recognition of articulated figures in terms of structural three-dimensional models.
Hogg, D. Model-based vision: a program to see a walking person. Image Vis. Comp. 1, 5–19 (1983).
Hoffman, D. D. & Flinchbaugh, B. E. The interpretation of biological motion. Biol. Cybern. 42, 195–204 (1982).
Webb, J. A. & Aggarwal, J. K. Structure from motion of rigid and jointed objects. Artif. Intell. 19, 107–130 (1982). A classical paper on the estimation of the structure of articulated objects based on geometrical constraints.
Oram, M. W. & Perrett, D. I. Neural processing of biological motion in the macaque temporal cortex. In Computational Vision Based on Neurobiology. (ed. Lawton, T. B.) SPIE Proc. 2054, 155–165 (1994).
Pinto, J. & Shiffrar, M. Subconfigurations of the human form in the perception of biological motion displays. Acta Psychol. 102, 293–318 (1999).
Cutting, J. E., Moore, C. & Morrison, R. Masking the motions of human gait. Percept. Psychophys. 44, 339–347 (1988). A psychophysical experiment showing that point-light walkers can be recognized even in the presence of a substantial number of moving background dots and scrambled walker noise.
Bertenthal, B. I. J. & Pinto, J. Global processing of biological motion. Psychol. Sci. 5, 221–225 (1994).
Thornton, I. M., Rensink, R. A. & Shiffrar, M. Active versus passive processing of biological motion. Perception 31, 837–853 (2002). A psychophysical study using a dual-task experiment to show that the tolerance of biological motion recognition against masking depends on attention.
McLeod, P., Dittrich, W., Driver, J., Perrett, D. & Zihl, J. Preserved and impaired detection of structure from motion by a 'motion-blind' patient. Vis. Cogn. 3, 363–391 (1996).
Vaina, L. M., Lemay, M., Bienfang, D. C., Choi, A. Y. & Nakayama, K. Intact 'biological motion' and 'structure from motion' perception in a patient with impaired motion mechanisms: a case study. Vis. Neurosci. 5, 353–369 (1990). References 120 and 121 describe studies of patients that have lesions in the dorsal pathway and that can still recognize biological motion.
Neri, P., Morrone, C. & Burr, D. C. Seeing biological motion. Nature 395, 894–896 (1998).
Logothetis, N. K., Pauls, J., Augath, M., Trinath, T. & Oeltermann, A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157 (2001).
Grill-Spector, K. & Malach, R. fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol. 107, 293–321 (2001).
Poggio, T. A theory of how the brain might work. Proc. Cold Spring Harb. Symp. Quant. Biol. 55, 899–910 (1990).
Wallis, G. & Rolls, E. T. Invariant face and object recognition in the visual system. Prog. Neurobiol. 51, 167 (1997).
Fox, R. & McDaniel, C. The perception of biological motion by human infants. Science 218, 486–487 (1982).
Cavanagh, P., Labianca, A. P. & Thornton, I. M. Attention-based visual routines: sprites. Cognition 80, 47–60 (2001). A psychophysical study using a visual search experiment to show that biological motion recognition in more complex tasks requires attention.
Shiffrar, M. & Freyd, J. J. Apparent motion of the human body. Psychol. Sci. 1, 257–264 (1993). A psychophysical investigation of the perceived paths of apparent motion between two photographs of human body configurations. For small ISIs, motion along the shortest paths is perceived, so the percept is dominated by low-level motion perception. For long ISIs, subjects perceive the biologically possible paths, indicating top–down influences from motion recognition.
Umiltà, M. A. et al. I know what you are doing: a neurophysiological study. Neuron 31, 155–165 (2001).
Verfaille, K. & Daems, A. Representing and anticipating actions in vision. Vis. Cogn. 9, 217–232 (2002).
Bassili, J. N. Facial motion in the perception of faces and of emotional expression. J. Exp. Psychol. Hum. Percept. Perform. 4, 373–379 (1978).
Poizner, H., Bellugi, U. & Lutes-Driscoll, V. Perception of American Sign Language in dynamic point-light display. J. Exp. Psychol. Hum. Percept. Perform. 7, 430–440 (1981).
Runeson, S. & Frykholm, G. Visual perception of lifted weight. J. Exp. Psychol. Hum. Percept. Perform. 7, 733–740 (1981).
Verfaillie, K. Perceiving human locomotion: priming effects in direction discrimination. Brain Cogn. 44, 192–213 (2000). A sound review of the experimental studies on biological motion perception that argues in favour of a view-based representation of biological motion patterns.
Goddard, N. H. The Perception of Articulated Motion: Recognizing Moving Light Displays. Thesis, Dept. of Computer Science, Univ. of Rochester (1992). A connectionist model for biological motion recognition that proposes a similar architecture to the model used in this review.
Little, J. J. & Boyd, J. Recognizing people by their gait: the shape of motion. J. Videre 1, 1–32 (1998).
Rosenblum, M., Yacoob, Y. & Davis, L. S. Human expression recognition from motion using a radial basis function network architecture. IEEE Trans. Neural Networks 7, 1121–1138 (1996).
Cedras, C. & Shah, M. Motion-based recognition: a survey. Image Vis. Comp. 13, 129–155 (1995).
Gavrila, D. M. The visual analysis of human movement: a survey. Comp. Vis. Image Underst. 73, 82–98 (1999).
Aggarwal, J. K. & Cai, Q. Human motion analysis: a review. Comp. Vis. Image Underst. 73, 428–440 (1999). References 139–141 review computer vision methods for the recognition of biological movements.
Song, Y., Goncalves, L., di Bernardo, E. & Perona, P. Monocular perception of biological motion in Johansson displays. Comp. Vis. Image Underst. 81, 303–327 (2001).
We thank R. Blake, D. Perrett, T. Sejnowski, C. Koch, S. Edelman, I. Bülthoff, E. Curio, Z. Kourtzi, M. Riesenhuber, P. Sinha, I. Thornton and L. Vaina for comments; A. Benali, Z. Kourtzi and C. Curio for help with data acquisition; and the Max-Planck Institute for Biological Cybernetics, Tübingen, for providing support. M.G. was supported by the Deutsche Forschungsgemeinschaft, Honda R&D Americas Inc. and the Deusche Volkswagen Stiftung. T.P. is supported in part by the Eugene McDermott chair. Research at CBCL was sponsored by the National Science Foundation. Additional support is provided by Central Research Institute of Electric Power Industry, Eastman Kodak Company, DaimlerChrysler AG, Compaq, Komatsu Ltd., NEC Fund, Nippon Telegraph & Telephone, Siemens Corporate Research Inc., Honda and Toyota.
Supplementary information: Movies that illustrate the stimuli that have been used in experiments to study the recognition of biological movements.
Movie 1 (MOV 626 kb)
Full-body stimulus walking
Full-body stimulus running (MOV 100 kb)
- BIOLOGICAL MOVEMENT
In psychophysics, 'biological motion' often refers specifically to point-light stimuli. We use the term 'biological movements' to characterize stimuli that show the movements of animals and humans, independent of the presentation mode.
- SUPERIOR TEMPORAL SULCUS
(STS). A sulcus in the temporal lobe of monkey and human cortex that contains areas that are selectively activated by biological movements.
Neurons that respond to sensory input in more than one modality, for example, both visual and auditory stimuli.
- OPTIC FLOW
A field of motion vectors that specifies how points of a frame (in an image sequence) are displaced over time. Unlike densely textured scenes, point-light stimuli do not specify a dense, spatially continuous, optic-flow field.
- NEURAL FEATURE DETECTORS
Neurons in cortex can often be interpreted as graded 'detectors' that are activated when specific features (such as orientation, corners, local motion with defined speed and direction, and faces) are present in their receptive fields.
- GABOR-LIKE FILTERS
Gabor functions are defined by sinusoidal functions that are windowed by a gaussian function. They define filters that are localized in the spatial domain as well as in the spatial frequency domain.
- MAXIMUM-LIKE OPERATION
An operation that results in an output signal that approximates the maximum among several input signals. Maximum computation can be approximated by physiologically plausible neural circuits.
- GAUSSIAN RADIAL BASIS FUNCTIONS
(RBF). Units used to model neurons that are tuned for complex stimuli. Their activation is described by a multi-dimensional gaussian function that depends on the difference between input signals and a constant vector (RBF centre) that defines the input that induces maximal activity.
- LEAKY INTEGRATOR
A simplified model for the dynamics of the membrane potential of a neuron. The dynamics are given by a linear differential equation.
- REICHARDT DETECTOR
A simple model for local motion detectors, originally studied in the eye of the beetle and the fly. The Reichardt detector multiplies the output signals of two receptors with different positions in the visual field, after delaying or low-pass filtering the output of one of them. The output of the detector is direction-selective. Versions of the Reichardt detector have been used to model direction-selective neurons in the primary visual cortex.
- MOTION MORPHING
An algorithm that continuously interpolates between different movement patterns, for example, walking and running. Good morphing algorithms result in interpolations that are very similar to natural movements.
The interval between any two frequencies that have a ratio of 2 to 1.
- HIDDEN MARKOV MODEL
A finite set of states, each of which is associated with a probability distribution. Transitions among the states are governed by a set of so-called transition probabilities. In a given state, an outcome can be generated according to the associated probability distribution. Only the outcome, not the state, is visible to an external observer and so the states are 'hidden'.
About this article
Cite this article
Giese, M., Poggio, T. Neural mechanisms for the recognition of biological movements. Nat Rev Neurosci 4, 179–192 (2003). https://doi.org/10.1038/nrn1057
This article is cited by
Nature Human Behaviour (2023)
Scientific Reports (2023)
Scientific Reports (2023)
BMC Neuroscience (2022)
A calibratable sensory neuron based on epitaxial VO2 for spike-based neuromorphic multisensory system
Nature Communications (2022)