Abstract
Humans exploit a range of visual depth cues to estimate three-dimensional structure. For example, the slant of a nearby tabletop can be judged by combining information from binocular disparity, texture and perspective. Behavioral tests show humans combine cues near-optimally, a feat that could depend on discriminating the outputs from cue-specific mechanisms or on fusing signals into a common representation. Although fusion is computationally attractive, it poses a substantial challenge, requiring the integration of quantitatively different signals. We used functional magnetic resonance imaging (fMRI) to provide evidence that dorsal visual area V3B/KO meets this challenge. Specifically, we found that fMRI responses are more discriminable when two cues (binocular disparity and relative motion) concurrently signal depth, and that information provided by one cue is diagnostic of depth indicated by the other. This suggests a cortical node important when perceiving depth, and highlights computations based on fusion in the dorsal stream.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Dosher, B.A., Sperling, G. & Wurst, S.A. Tradeoffs between stereopsis and proximity luminance covariance as determinants of perceived 3D structure. Vision Res. 26, 973–990 (1986).
Bülthoff, H.H. & Mallot, H.A. Integration of depth modules – stereo and shading. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 5, 1749–1758 (1988).
Landy, M.S., Maloney, L.T., Johnston, E.B. & Young, M. Measurement and modeling of depth cue combination – in defense of weak fusion. Vision Res. 35, 389–412 (1995).
Clark, J.J. & Yuille, A.L. Data Fusion for Sensory Information Processing Systems (Kluwer Academic, 1990).
Ernst, M.O. & Banks, M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433 (2002).
Knill, D.C. & Saunders, J.A. Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Res. 43, 2539–2558 (2003).
Tsutsui, K., Sakata, H., Naganuma, T. & Taira, M. Neural correlates for perception of 3D surface orientation from texture gradient. Science 298, 409–412 (2002).
Nadler, J.W., Angelaki, D.E. & DeAngelis, G.C. A neural representation of depth from motion parallax in macaque visual cortex. Nature 452, 642–645 (2008).
Liu, Y., Vogels, R. & Orban, G.A. Convergence of depth from texture and depth from disparity in macaque inferior temporal cortex. J. Neurosci. 24, 3795–3800 (2004).
Gu, Y., Angelaki, D.E. & DeAngelis, G.C. Neural correlates of multisensory cue integration in macaque MSTd. Nat. Neurosci. 11, 1201–1210 (2008).
Morgan, M.L., DeAngelis, G.C. & Angelaki, D.E. Multisensory integration in macaque visual cortex depends on cue reliability. Neuron 59, 662–673 (2008).
Rogers, B. & Graham, M. Motion parallax as an independent cue for depth perception. Perception 8, 125–134 (1979).
Bradshaw, M.F. & Rogers, B.J. The interaction of binocular disparity and motion parallax in the computation of depth. Vision Res. 36, 3457–3468 (1996).
Nawrot, M. & Blake, R. Neural integration of information specifying structure from stereopsis and motion. Science 244, 716–718 (1989).
Poom, L. & Borjesson, E. Perceptual depth synthesis in the visual system as revealed by selective adaptation. J. Exp. Psychol. Hum. Percept. Perform. 25, 504–517 (1999).
Domini, F., Caudek, C. & Tassinari, H. Stereo and motion information are not independently processed by the visual system. Vision Res. 46, 1707–1723 (2006).
DeAngelis, G.C. & Newsome, W.T. Organization of disparity-selective neurons in macaque area MT. J. Neurosci. 19, 1398–1415 (1999).
Preston, T.J., Li, S., Kourtzi, Z. & Welchman, A.E. Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain. J. Neurosci. 28, 11315–11327 (2008).
Nandy, A.S. & Tjan, B.S. Efficient integration across spatial frequencies for letter identification in foveal and peripheral vision. J. Vis. 8, 3 (2008).
Hillis, J.M., Ernst, M.O., Banks, M.S. & Landy, M.S. Combining sensory information: mandatory fusion within, but not between, senses. Science 298, 1627–1630 (2002).
Körding, K.P. et al. Causal inference in multisensory perception. PLoS ONE 2, e943 (2007).
Popple, A.V., Smallman, H.S. & Findlay, J.M. The area of spatial integration for initial horizontal disparity vergence. Vision Res. 38, 319–326 (1998).
Preston, T.J., Kourtzi, Z. & Welchman, A.E. Adaptive estimation of three-dimensional structure in the human brain. J. Neurosci. 29, 1688–1698 (2009).
Tyler, C.W., Likova, L.T., Kontsevich, L.L. & Wade, A.R. The specificity of cortical region KO to depth structure. Neuroimage 30, 228–238 (2006).
Meredith, M.A. & Stein, B.E. Interactions among converging sensory inputs in the superior colliculus. Science 221, 389–391 (1983).
Avillac, M., Ben Hamed, S. & Duhamel, J.R. Multisensory integration in the ventral intraparietal area of the macaque monkey. J. Neurosci. 27, 1922–1932 (2007).
Stanford, T.R., Quessy, S. & Stein, B.E. Evaluating the operations underlying multisensory integration in the cat superior colliculus. J. Neurosci. 25, 6499–6508 (2005).
Ma, W.J., Beck, J.M., Latham, P.E. & Pouget, A. Bayesian inference with probabilistic population codes. Nat. Neurosci. 9, 1432–1438 (2006).
Orban, G.A., Janssen, P. & Vogels, R. Extracting 3D structure from disparity. Trends Neurosci. 29, 466–473 (2006).
Parker, A.J. Binocular depth perception and the cerebral cortex. Nat. Rev. Neurosci. 8, 379–391 (2007).
Backus, B.T., Fleet, D.J., Parker, A.J. & Heeger, D.J. Human cortical activity correlates with stereoscopic depth perception. J. Neurophysiol. 86, 2054–2068 (2001).
Chandrasekaran, C., Canon, V., Dahmen, J.C., Kourtzi, Z. & Welchman, A.E. Neural correlates of disparity-defined shape discrimination in the human brain. J. Neurophysiol. 97, 1553–1565 (2007).
Orban, G.A., Sunaert, S., Todd, J.T., Van Hecke, P. & Marchal, G. Human cortical regions involved in extracting depth from motion. Neuron 24, 929–940 (1999).
Murray, S.O., Olshausen, B.A. & Woods, D.L. Processing shape, motion and three-dimensional shape-from-motion in the human cortex. Cereb. Cortex 13, 508–516 (2003).
Paradis, A.L. et al. Visual perception of motion and 3-D structure from motion: an fMRI study. Cereb. Cortex 10, 772–783 (2000).
Sereno, M.E., Trinath, T., Augath, M. & Logothetis, N.K. Three-dimensional shape representation in monkey cortex. Neuron 33, 635–652 (2002).
Durand, J.B. et al. Anterior regions of monkey parietal cortex process visual 3D shape. Neuron 55, 493–505 (2007).
Peuskens, H. et al. Attention to 3-D shape, 3-D motion, and texture in 3-D structure from motion displays. J. Cogn. Neurosci. 16, 665–682 (2004).
Orban, G.A. et al. Similarities and differences in motion processing between the human and macaque brain: evidence from fMRI. Neuropsychologia 41, 1757–1768 (2003).
Shmuel, A., Chaimow, D., Raddatz, G., Ugurbil, K. & Yacoub, E. Mechanisms underlying decoding at 7 T: ocular dominance columns, broad structures, and macroscopic blood vessels in V1 convey information on the stimulated eye. Neuroimage 49, 1957–1964 (2010).
Kriegeskorte, N., Cusack, R. & Bandettini, P. How does an fMRI voxel sample the neuronal activity pattern: compact-kernel or complex spatiotemporal filter? Neuroimage 49, 1965–1976 (2010).
Op de Beeck, H.P. Against hyperacuity in brain reading: spatial smoothing does not hurt multivariate fMRI analyses? Neuroimage 49, 1943–1948 (2010).
Freeman, J., Brouwer, G.J., Heeger, D.J. & Merriam, E.P. Orientation decoding depends on maps, not columns. J. Neurosci. 31, 4792–4804 (2011).
Welchman, A.E., Deubelius, A., Conrad, V., Bülthoff, H.H. & Kourtzi, Z. 3D shape perception from combined depth cues in human visual cortex. Nat. Neurosci. 8, 820–827 (2005).
Tjan, B.S., Lestou, V. & Kourtzi, Z. Uncertainty and invariance in the human visual cortex. J. Neurophysiol. 96, 1556–1568 (2006).
Dupont, P. et al. The kinetic occipital region in human visual cortex. Cereb. Cortex 7, 283–292 (1997).
Serences, J.T. & Boynton, G.M. The representation of behavioral choice for motion in human visual cortex. J. Neurosci. 27, 12893–12899 (2007).
De Martino, F. et al. Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns. Neuroimage 43, 44–58 (2008).
Kamitani, Y. & Tong, F. Decoding the visual and subjective contents of the human brain. Nat. Neurosci. 8, 679–685 (2005).
Acknowledgements
We thank B. Tjan, R. Fleming and A. Glennerster for discussions on the project. The work was supported by fellowships to A.E.W. from the Wellcome Trust (095183/Z/10/Z) and Biotechnology and Biological Sciences Research Council (C520620) and to H.B. from the Japan Society for the Promotion of Science (H22,290).
Author information
Authors and Affiliations
Contributions
H.B. collected data, programmed stimuli, performed the analysis, wrote the simulations and prepared the work for publication; T.J.P. collected data, programmed stimuli and performed preliminary analysis; A.M. wrote MVPA analysis tools; A.E.W. originated and designed the study, collected data, performed and guided analysis, wrote the simulations, prepared the work for publication and wrote the paper.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–7 (PDF 4828 kb)
Rights and permissions
About this article
Cite this article
Ban, H., Preston, T., Meeson, A. et al. The integration of motion and disparity cues to depth in dorsal visual cortex. Nat Neurosci 15, 636–643 (2012). https://doi.org/10.1038/nn.3046
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.3046
This article is cited by
-
The effect of different depth planes during a manual tracking task in three-dimensional virtual reality space
Scientific Reports (2023)
-
Human primary visual cortex shows larger population receptive fields for binocular disparity-defined stimuli
Brain Structure and Function (2021)
-
Merging familiar and new senses to perceive and act in space
Cognitive Processing (2021)
-
White matter dissection and structural connectivity of the human vertical occipital fasciculus to link vision-associated brain cortex
Scientific Reports (2020)
-
Image Segmentation Based on Relative Motion and Relative Disparity Cues in Topographically Organized Areas of Human Visual Cortex
Scientific Reports (2019)