Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Sensors for impossible stimuli may solve the stereo correspondence problem

Nature Neuroscience volume 10, pages 1322–1328 (2007)Cite this article

Abstract

One of the fundamental challenges of binocular vision is that objects project to different positions on the two retinas (binocular disparity). Neurons in visual cortex show two distinct types of tuning to disparity, position and phase disparity, which are the results of differences in receptive field location and profile, respectively. Here, we point out that phase disparity does not occur in natural images. Why, then, should the brain encode it? We propose that phase-disparity detectors help to work out which feature in the left eye corresponds to a given feature in the right. This correspondence problem is plagued by false matches: regions of the image that look similar, but do not correspond to the same object. We show that phase-disparity neurons tend to be more strongly activated by false matches. Thus, they may act as 'lie detectors', enabling the true correspondence to be deduced by a process of elimination.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Different types of disparity.
Figure 2: Response of a neuronal population to a broadband image with uniform disparity of 0.06°.
Figure 3: Comparison of our algorithm with four possible implementations of a maximum-energy algorithm, tested on a uniform-disparity noise stimulus.
Figure 4: Sketch of the algorithm used to estimate stimulus disparity in a single channel.
Figure 5: Applying our principle to a classic test stereogram.
Figure 6: (a–c) Three test stereo-pairs from the Middlebury repository.

Similar content being viewed by others

References

  1. Ohzawa, I., DeAngelis, G.C. & Freeman, R.D. Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. Science 249, 1037–1041 (1990).

    Article  CAS  Google Scholar 

  2. DeAngelis, G.C., Ohzawa, I. & Freeman, R.D. Depth is encoded in the visual cortex by a specialised receptive field structure. Nature 352, 156–159 (1991).

    Article  CAS  Google Scholar 

  3. Anzai, A., Ohzawa, I. & Freeman, R.D. Neural mechanisms underlying binocular fusion and stereopsis: position vs. phase. Proc. Natl. Acad. Sci. USA 94, 5438–5443 (1997).

    Article  CAS  Google Scholar 

  4. Anzai, A., Ohzawa, I. & Freeman, R.D. Neural mechanisms for encoding binocular disparity: receptive field position versus phase. J. Neurophysiol. 82, 874–890 (1999).

    Article  CAS  Google Scholar 

  5. Livingstone, M.S. & Tsao, D.Y. Receptive fields of disparity-selective neurons in macaque striate cortex. Nat. Neurosci. 2, 825–832 (1999).

    Article  CAS  Google Scholar 

  6. Prince, S.J., Cumming, B.G. & Parker, A.J. Range and mechanism of encoding of horizontal disparity in macaque V1. J. Neurophysiol. 87, 209–221 (2002).

    Article  CAS  Google Scholar 

  7. Fleet, D.J., Wagner, H. & Heeger, D.J. Neural encoding of binocular disparity: energy models, position shifts and phase shifts. Vision Res. 36, 1839–1857 (1996).

    Article  CAS  Google Scholar 

  8. Orban, G.A., Janssen, P. & Vogels, R. Extracting 3D structure from disparity. Trends Neurosci. 29, 466–473 (2006).

    Article  CAS  Google Scholar 

  9. Bredfeldt, C.E. & Cumming, B.G. A simple account of cyclopean edge responses in macaque V2. J. Neurosci. 26, 7581–7596 (2006).

    Article  CAS  Google Scholar 

  10. Nienborg, H., Bridge, H., Parker, A.J. & Cumming, B.G. Receptive field size in V1 neurons limits acuity for perceiving disparity modulation. J. Neurosci. 24, 2065–2076 (2004).

    Article  CAS  Google Scholar 

  11. von der Heydt, R., Zhou, H. & Friedman, H.S. Representation of stereoscopic edges in monkey visual cortex. Vision Res. 40, 1955–1967 (2000).

    Article  CAS  Google Scholar 

  12. Hinkle, D.A. & Connor, C.E. Three-dimensional orientation tuning in macaque area V4. Nat. Neurosci. 5, 665–670 (2002).

    Article  CAS  Google Scholar 

  13. Nguyenkim, J.D. & DeAngelis, G.C. Disparity-based coding of three-dimensional surface orientation by macaque middle temporal neurons. J. Neurosci. 23, 7117–7128 (2003).

    Article  CAS  Google Scholar 

  14. Janssen, P., Vogels, R. & Orban, G.A. Three-dimensional shape coding in inferior temporal cortex. Neuron 27, 385–397 (2000).

    Article  CAS  Google Scholar 

  15. Taira, M., Tsutsui, K.I., Jiang, M., Yara, K. & Sakata, H. Parietal neurons represent surface orientation from the gradient of binocular disparity. J. Neurophysiol. 83, 3140–3146 (2000).

    Article  CAS  Google Scholar 

  16. Tyler, C.W. Depth perception in disparity gratings. Nature 251, 140–142 (1974).

    Article  CAS  Google Scholar 

  17. Prince, S.J., Eagle, R.A. & Rogers, B.J. Contrast masking reveals spatial-frequency channels in stereopsis. Perception 27, 1345–1355 (1998).

    Article  CAS  Google Scholar 

  18. Haefner, R. & Cumming, B.G. Spatial nonlinearities in V1 disparity-selective neurons. Soc. Neurosci. Abstr. 583.9 (2005).

  19. Read, J.C.A. A Bayesian model of stereopsis depth and motion direction discrimination. Biol. Cybern. 86, 117–136 (2002).

    Article  CAS  Google Scholar 

  20. Read, J.C.A. A Bayesian approach to the stereo correspondence problem. Neural Comput. 14, 1371–1392 (2002).

    Article  Google Scholar 

  21. Tsai, J.J. & Victor, J.D. Reading a population code: a multi-scale neural model for representing binocular disparity. Vision Res. 43, 445–466 (2003).

    Article  Google Scholar 

  22. Chen, Y. & Qian, N. A coarse-to-fine disparity energy model with both phase-shift and position-shift receptive field mechanisms. Neural Comput. 16, 1545–1577 (2004).

    Article  Google Scholar 

  23. Read, J.C.A. & Cumming, B.G. Ocular dominance predicts neither strength nor class of disparity selectivity with random-dot stimuli in primate V1. J. Neurophysiol. 91, 1271–1281 (2004).

    Article  Google Scholar 

  24. Cumming, B.G. & DeAngelis, G.C. The physiology of stereopsis. Annu. Rev. Neurosci. 24, 203–238 (2001).

    Article  CAS  Google Scholar 

  25. Freeman, R.D. & Ohzawa, I. On the neurophysiological organisation of binocular vision. Vision Res. 30, 1661–1676 (1990).

    Article  CAS  Google Scholar 

  26. Poggio, G.F. & Fischer, B. Binocular interaction and depth sensitivity of striate and prestriate cortex of behaving rhesus monkey. J. Neurophysiol. 40, 1392–1405 (1977).

    Article  CAS  Google Scholar 

  27. Qian, N. Computing stereo disparity and motion with known binocular cell properties. Neural Comput. 6, 390–404 (1994).

    Article  Google Scholar 

  28. Poggio, G.F., Motter, B.C., Squatrito, S. & Trotter, Y. Responses of neurons in visual cortex (V1 and V2) of the alert macaque to dynamic random-dot stereograms. Vision Res. 25, 397–406 (1985).

    Article  CAS  Google Scholar 

  29. Qian, N. & Zhu, Y. Physiological computation of binocular disparity. Vision Res. 37, 1811–1827 (1997).

    Article  CAS  Google Scholar 

  30. Qian, N. Binocular disparity and the perception of depth. Neuron 18, 359–368 (1997).

    Article  CAS  Google Scholar 

  31. Sanger, T. Stereo disparity computation using Gabor filters. Biol. Cybern. 59, 405–418 (1988).

    Article  Google Scholar 

  32. Marr, D. & Poggio, T. A computational theory of human stereo vision. Proc. R. Soc. Lond. B 204, 301–328 (1979).

    Article  CAS  Google Scholar 

  33. Smallman, H.S. & MacLeod, D.I. Spatial scale interactions in stereo sensitivity and the neural representation of binocular disparity. Perception 26, 977–994 (1997).

    Article  CAS  Google Scholar 

  34. Rohaly, A.M. & Wilson, H.R. Nature of coarse-to-fine constraints on binocular fusion. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 10, 2433–2441 (1993).

    Article  CAS  Google Scholar 

  35. Prince, S.J. & Eagle, R.A. Size-disparity correlation in human binocular depth perception. Proc. Biol. Soc. 266, 1361–1365 (1999).

    Article  CAS  Google Scholar 

  36. Prince, S.J.P. & Eagle, R.E. Weighted directional energy model of human stereo correspondence. Vision Res. 40, 1143–1155 (2000).

    Article  CAS  Google Scholar 

  37. Read, J.C.A. & Cumming, B.G. Testing quantitative models of binocular disparity selectivity in primary visual cortex. J. Neurophysiol. 90, 2795–2817 (2003).

    Article  Google Scholar 

  38. DeValois, R.L., Albrecht, D.G. & Thorell, L.G. Spatial frequency selectivity of cells in macaque visual cortex. Vision Res. 22, 545–559 (1982).

    Article  CAS  Google Scholar 

  39. Cumming, B.G., Shapiro, S.E. & Parker, A.J. Disparity detection in anticorrelated stereograms. Perception 27, 1367–1377 (1998).

    Article  CAS  Google Scholar 

  40. Cogan, A.I., Lomakin, A.J. & Rossi, A.F. Depth in anticorrelated stereograms: effects of spatial density and interocular delay. Vision Res. 33, 1959–1975 (1993).

    Article  CAS  Google Scholar 

  41. Julesz, B. Foundations of Cyclopean Perception (University of Chicago Press, Chicago, 1971).

    Google Scholar 

  42. Rogers, B. & Anstis, S. Reversed depth from positive and negative stereograms. Perception 4, 193–201 (1975).

    Article  Google Scholar 

  43. Scharstein, D. & Szeliski, R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002).

    Article  Google Scholar 

  44. Parker, A. Misaligned viewpoints. Nature 352, 109 (1991).

    Article  CAS  Google Scholar 

  45. Jepson, A.D. & Jenkin, M.R.M. The fast computation of disparity from phase differences. in Computer Vision and Pattern Recognition, 1989. Proceedings CVPR '89., IEEE Computer Society Conference on., 389–403 (San Diego, California, USA, 1989).

    Google Scholar 

  46. Fleet, D.J. Disparity from local weighted phase-correlation. in Systems, Man, and Cybernetics, 1994. 'Humans, Information and Technology'., 1994 IEEE International Conference on. 1, 48–54 (San Antonio, Texas, USA, 1994).

    Google Scholar 

Download references

Acknowledgements

J.C.A.R. performed all analyses and simulations and wrote the manuscript. B.G.C. supervised the project.

Author information

Authors and Affiliations

  1. Institute of Neuroscience, Newcastle University, Framlington Place, Newcastle upon Tyne, NE2 4HH, UK

    Jenny C A Read

  2. Laboratory of Sensorimotor Research, National Eye Institute, 49 Convent Drive, US National Institutes of Health, Bethesda, 20892-4435, Maryland, USA

    Bruce G Cumming

Authors
  1. Jenny C A Read
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruce G Cumming
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jenny C A Read.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–6, Supplementary Note (PDF 6781 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Read, J., Cumming, B. Sensors for impossible stimuli may solve the stereo correspondence problem. Nat Neurosci 10, 1322–1328 (2007). https://doi.org/10.1038/nn1951

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn1951

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing