A neural representation of depth from motion parallax in macaque visual cortex

Abstract

Perception of depth is a fundamental challenge for the visual system, particularly for observers moving through their environment. The brain makes use of multiple visual cues to reconstruct the three-dimensional structure of a scene. One potent cue, motion parallax, frequently arises during translation of the observer because the images of objects at different distances move across the retina with different velocities. Human psychophysical studies have demonstrated that motion parallax can be a powerful depth cue1,2,3,4,5, and motion parallax seems to be heavily exploited by animal species that lack highly developed binocular vision6,7,8. However, little is known about the neural mechanisms that underlie this capacity. Here we show, by using a virtual-reality system to translate macaque monkeys (Macaca mulatta) while they viewed motion parallax displays that simulated objects at different depths, that many neurons in the middle temporal area (area MT) signal the sign of depth (near versus far) from motion parallax in the absence of other depth cues. To achieve this, neurons must combine visual motion with extra-retinal (non-visual) signals related to the animal’s movement. Our findings suggest a new neural substrate for depth perception and demonstrate a robust interaction of visual and non-visual cues in area MT. Combined with previous studies that implicate area MT in depth perception based on binocular disparities9,10,11,12, our results suggest that area MT contains a more general representation of three-dimensional space that makes use of multiple cues.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Schematic illustration of motion parallax and stimulus design.
Figure 2: A neuron selective for depth from motion parallax.
Figure 3: Tuning curves from six additional MT cells.
Figure 4: Many MT neurons are selective for depth defined by motion parallax.

References

  1. 1

    Rogers, B. & Graham, M. Motion parallax as an independent cue for depth perception. Perception 8, 125–134 (1979)

    CAS  Article  Google Scholar 

  2. 2

    Rogers, B. J. Motion parallax and other dynamic cues for depth in humans. Rev. Oculomot. Res. 5, 119–137 (1993)

    ADS  CAS  PubMed  Google Scholar 

  3. 3

    Rogers, B. J. & Collett, T. S. The appearance of surfaces specified by motion parallax and binocular disparity. Q. J. Exp. Psychol. A 41, 697–717 (1989)

    CAS  Article  Google Scholar 

  4. 4

    Rogers, B. J. & Graham, M. E. Similarities between motion parallax and stereopsis in human depth perception. Vision Res. 22, 261–270 (1982)

    CAS  Article  Google Scholar 

  5. 5

    Bradshaw, M. F. & Rogers, B. J. The interaction of binocular disparity and motion parallax in the computation of depth. Vision Res. 36, 3457–3468 (1996)

    CAS  Article  Google Scholar 

  6. 6

    Ellard, C. G., Goodale, M. A. & Timney, B. Distance estimation in the Mongolian gerbil: the role of dynamic depth cues. Behav. Brain Res. 14, 29–39 (1984)

    CAS  Article  Google Scholar 

  7. 7

    van der Willigen, R. F., Frost, B. J. & Wagner, H. Depth generalization from stereo to motion parallax in the owl. J. Comp. Physiol. A 187, 997–1007 (2002)

    Article  Google Scholar 

  8. 8

    Kral, K. Behavioural–analytical studies of the role of head movements in depth perception in insects, birds and mammals. Behav. Processes 64, 1–12 (2003)

    Article  Google Scholar 

  9. 9

    DeAngelis, G. C., Cumming, B. G. & Newsome, W. T. Cortical area MT and the perception of stereoscopic depth. Nature 394, 677–680 (1998)

    ADS  CAS  Article  Google Scholar 

  10. 10

    Uka, T. & DeAngelis, G. C. Contribution of middle temporal area to coarse depth discrimination: comparison of neuronal and psychophysical sensitivity. J. Neurosci. 23, 3515–3530 (2003)

    CAS  Article  Google Scholar 

  11. 11

    Uka, T. & DeAngelis, G. C. Linking neural representation to function in stereoscopic depth perception: roles of the middle temporal area in coarse versus fine disparity discrimination. J. Neurosci. 26, 6791–6802 (2006)

    CAS  Article  Google Scholar 

  12. 12

    Uka, T. & DeAngelis, G. C. Contribution of area MT to stereoscopic depth perception: choice-related response modulations reflect task strategy. Neuron 42, 297–310 (2004)

    CAS  Article  Google Scholar 

  13. 13

    Farber, J. M. & McConkie, A. B. Optical motions as information for unsigned depth. J. Exp. Psychol. Hum. Percept. Perform. 5, 494–500 (1979)

    CAS  Article  Google Scholar 

  14. 14

    Rogers, S. & Rogers, B. J. Visual and nonvisual information disambiguate surfaces specified by motion parallax. Percept. Psychophys. 52, 446–452 (1992)

    CAS  Article  Google Scholar 

  15. 15

    Nawrot, M. Eye movements provide the extra-retinal signal required for the perception of depth from motion parallax. Vision Res. 43, 1553–1562 (2003a)

    Article  Google Scholar 

  16. 16

    DeAngelis, G. C. & Uka, T. Coding of horizontal disparity and velocity by MT neurons in the alert macaque. J. Neurophysiol. 89, 1094–1111 (2003)

    Article  Google Scholar 

  17. 17

    Maunsell, J. H. & Van Essen, D. C. Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. J. Neurophysiol. 49, 1148–1167 (1983b)

    CAS  Article  Google Scholar 

  18. 18

    Maunsell, J. H. & Van Essen, D. C. Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. J. Neurophysiol. 49, 1127–1147 (1983a)

    CAS  Article  Google Scholar 

  19. 19

    Nover, H., Anderson, C. H. & DeAngelis, G. C. A logarithmic, scale-invariant representation of speed in macaque middle temporal area accounts for speed discriminiation performance. J. Neurosci. 25, 10049–10060 (2005)

    CAS  Article  Google Scholar 

  20. 20

    Born, R. T. & Bradley, D. C. Structure and function of visual area MT. Annu. Rev. Neurosci. 28, 157–189 (2005)

    CAS  Article  Google Scholar 

  21. 21

    Bradley, D. C., Qian, N. & Andersen, R. A. Integration of motion and stereopsis in middle temporal cortical area of macaques. Nature 373, 609–611 (1995)

    ADS  CAS  Article  Google Scholar 

  22. 22

    Bradley, D. C., Chang, G. C. & Andersen, R. A. Encoding of three-dimensional structure-from-motion by primate area MT neurons. Nature 392, 714–717 (1998)

    ADS  CAS  Article  Google Scholar 

  23. 23

    Dodd, J. V., Krug, K., Cumming, B. G. & Parker, A. J. Perceptually bistable three-dimensional figures evoke high choice probabilities in cortical area MT. J. Neurosci. 21, 4809–4821 (2001)

    CAS  Article  Google Scholar 

  24. 24

    Cao, A. & Schiller, P. H. Neural responses to relative speed in the primary visual cortex of rhesus monkey. Vis. Neurosci. 20, 77–84 (2003)

    Article  Google Scholar 

  25. 25

    Newsome, W. T., Wurtz, R. H. & Komatsu, H. Relation of cortical areas MT and MST to pursuit eye movements. II. Differentiation of retinal from extraretinal inputs. J. Neurophysiol. 60, 604–620 (1988)

    CAS  Article  Google Scholar 

  26. 26

    Bremmer, F., Ilg, U. J., Thiele, A., Distler, C. & Hoffmann, K. P. Eye position effects in monkey cortex. I. Visual and pursuit-related activity in extrastriate areas MT and MST. J. Neurophysiol. 77, 944–961 (1997)

    CAS  Article  Google Scholar 

  27. 27

    Nawrot, M. & Joyce, L. The pursuit theory of motion parallax. Vision Res. 46, 4709–4725 (2006)

    Article  Google Scholar 

  28. 28

    Gu, Y., Watkins, P. V., Angelaki, D. E. & DeAngelis, G. C. Visual and nonvisual contributions to three-dimensional heading selectivity in the medial superior temporal area. J. Neurosci. 26, 73–85 (2006)

    CAS  Article  Google Scholar 

  29. 29

    Gu, Y., DeAngelis, G. C. & Angelaki, D. E. A functional link between area MSTd and heading perception based on vestibular signals. Nature Neurosci. 10, 1038–1047 (2007)

    CAS  Article  Google Scholar 

  30. 30

    Nguyenkim, J. D. & DeAngelis, G. C. Disparity-based coding of three-dimensional surface orientation by macaque middle temporal neurons. J. Neurosci. 23, 7117–7128 (2003)

    CAS  Article  Google Scholar 

Download references

Acknowledgements

We thank L. Snyder, A. Anzai, T. Sanada, Y. Gu and C. Fetsch for comments; C. Broussard for technical development; and A. Turner, E. White and K. Kocher for care and training of monkeys. This work was supported by a National Eye Institute (NEI) institutional National Research Service Award (to J.W.N.) and NEI grants to G.C.D. and D.E.A.

Author Contributions J.W.N. and G.C.D. designed the stimuli; J.W.N. collected the data; J.W.N., D.E.A. and G.C.D. refined the analysis and presentation of the data; J.W.N., D.E.A. and G.C.D. wrote the paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Gregory C. DeAngelis.

Supplementary information

Supplementary Figures

The file contains Supplementary Figures S1-S7 with Legends. (PDF 367 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Nadler, J., Angelaki, D. & DeAngelis, G. A neural representation of depth from motion parallax in macaque visual cortex. Nature 452, 642–645 (2008). https://doi.org/10.1038/nature06814

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing