Article | Published:

Behavior and neural basis of near-optimal visual search

Nature Neuroscience volume 14, pages 783790 (2011) | Download Citation

Abstract

The ability to search efficiently for a target in a cluttered environment is one of the most remarkable functions of the nervous system. This task is difficult under natural circumstances, as the reliability of sensory information can vary greatly across space and time and is typically a priori unknown to the observer. In contrast, visual-search experiments commonly use stimuli of equal and known reliability. In a target detection task, we randomly assigned high or low reliability to each item on a trial-by-trial basis. An optimal observer would weight the observations by their trial-to-trial reliability and combine them using a specific nonlinear integration rule. We found that humans were near-optimal, regardless of whether distractors were homogeneous or heterogeneous and whether reliability was manipulated through contrast or shape. We present a neural-network implementation of near-optimal visual search based on probabilistic population coding. The network matched human performance.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    , & Measuring the effect of attention on simple visual search. J. Exp. Psychol. Hum. Percept. Perform. 19, 108–130 (1993).

  2. 2.

    & A feature-integration theory of attention. Cognit. Psychol. 12, 97–136 (1980).

  3. 3.

    & A detection method and probabilistic models for assessing information processing from brief visual displays. Proc. Natl. Acad. Sci. USA 52, 446–454 (1964).

  4. 4.

    Identifying attentional and decision-making components in information processing. in Attention and Performance (ed. R.S. Nickerson) 277–296 (Erlbaum, Hillsdale, New Jersey, 1980).

  5. 5.

    & Visual search for simple targets. Psychol. Bull. 81, 15–28 (1974).

  6. 6.

    , & The psychophysics of visual search. Vision Res. 40, 1227–1268 (2000).

  7. 7.

    & Visual search and stimulus similarity. Psychol. Rev. 96, 433–458 (1989).

  8. 8.

    Visual search for orientation among heterogeneous distractors: experimental results and implications for signal detection theory models of search. J. Exp. Psychol. Hum. Percept. Perform. 27, 985–999 (2001).

  9. 9.

    Object location in a complex perceptual field. J. Exp. Psychol. 45, 126–132 (1953).

  10. 10.

    & Visual search through color displays: effects of target-background similarity and background uniformity. Percept. Psychophys. 27, 267–272 (1980).

  11. 11.

    & Perception as Bayesian Inference (Cambridge University Press, New York, 1996).

  12. 12.

    & The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712–719 (2004).

  13. 13.

    , , & Bayesian inference with probabilistic population codes. Nat. Neurosci. 9, 1432–1438 (2006).

  14. 14.

    , & Multisensory integration in macaque visual cortex depends on cue reliability. Neuron 59, 662–673 (2008).

  15. 15.

    Visual search and attention: a signal detection theory approach. Neuron 31, 523–535 (2001).

  16. 16.

    , , & Statistical decision theory to relate neurons to behavior in the study of covert visual attention. Vision Res. 49, 1097–1128 (2009).

  17. 17.

    & Optimal eye movement strategies in visual search. Nature 434, 387–391 (2005).

  18. 18.

    , , & A signal detection model predicts the effects of set size on visual search accuracy for feature, conjunction, triple conjunction, and disjunction displays. Percept. Psychophys. 62, 425–451 (2000).

  19. 19.

    & Signal Detection Theory and Psychophysics (John Wiley & Sons, Los Altos, California, 1966).

  20. 20.

    , & The theory of signal detectability. IRE Prof. Group Inf. Theory 4, 171–212 (1954).

  21. 21.

    & More on the detection of one of M orthogonal signals. J. Acoust. Soc. Am. 41, 497–505 (1967).

  22. 22.

    The lower visual search efficiency for conjunctions is due to noise and not serial attentional processing. Psychol. Sci. 9, 111–118 (1998).

  23. 23.

    , & Signal dection models for multidimensional stimuli: probability distributions and combination rules. J. Math. Psychol. 31, 366–409 (1987).

  24. 24.

    A vector-magnitude model of contrast detection. Kybernetik 16, 65–67 (1974).

  25. 25.

    , & Inference and Computation with Population Codes. Annu. Rev. Neurosci. 26, 381–410 (2003).

  26. 26.

    , , , & Eye position effects in monkey cortex. I. Visual and pursuit-related activity in extrastriate areas MT and MST. J. Neurophysiol. 77, 944–961 (1997).

  27. 27.

    , & Encoding of spatial location by posterior parietal neurons. Science 230, 456–458 (1985).

  28. 28.

    Normalization of cell responses in cat striate cortex. Vis. Neurosci. 9, 181–197 (1992).

  29. 29.

    & The normalization model of attention. Neuron 61, 168–185 (2009).

  30. 30.

    , & Complex Bayesian inference in neural circuits using divisive normalization. Front. Syst. Neurosci. Conference Abstract: Computational and Systems Neuroscience 2009, doi: 10.3389/conf.neuro.06.2009.03.109 (2 February 2009).

  31. 31.

    & Simple model for reading neuronal population codes. Proc. Natl. Acad. Sci. USA 90, 10749–10753 (1993).

  32. 32.

    , , & Practising orientation identification improves orientation coding in V1 neurons. Nature 412, 549–553 (2001).

  33. 33.

    & Spatial–frequency discrimination and detection: comparison of postadaptation thresholds. J. Opt. Soc. Am. 73, 1684–1690 (1983).

  34. 34.

    & Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324, 759–764 (2009).

  35. 35.

    & Saccade target selection in the superior colliculus: a signal detection theory approach. J. Neurosci. 28, 2991–3007 (2008).

  36. 36.

    , , & Neural correlates, computation and behavioural impact of decision confidence. Nature 455, 227–231 (2008).

  37. 37.

    , & Reading population codes: a neural implementation of ideal observers. Nat. Neurosci. 2, 740–745 (1999).

  38. 38.

    & Target selection in area V4 during a multidimensional visual search task. J. Neurosci. 24, 6371–6382 (2004).

  39. 39.

    , & Parallel and serial neural mechanisms for visual search in macaque area V4. Science 308, 529–534 (2005).

  40. 40.

    , & The representation of visual salience in monkey parietal cortex. Nature 391, 481–484 (1998).

  41. 41.

    Mixture models and the probabilistic structure of depth cues. Vision Res. 43, 831–854 (2003).

  42. 42.

    , & Object perception as Bayesian inference. Annu. Rev. Psychol. 55, 271–304 (2004).

  43. 43.

    & Computational modeling of visual attention. Nat. Rev. Neurosci. 2, 194–203 (2001).

  44. 44.

    & Search goal tunes visual features optimally. Neuron 53, 605–617 (2007).

  45. 45.

    , , , & SUN: a Bayesian framework for saliency using natural statistics. J. Vis. 8, 1–20 (2008).

  46. 46.

    , , & Optimal feature integration in visual search. J. Vis. 9, 1–11 (2009).

  47. 47.

    et al. Bayesian decision-making with probabilistic population codes. Neuron 60, 1142–1152 (2008).

  48. 48.

    The Psychophysics Toolbox. Spat. Vis. 10, 433–436 (1997).

  49. 49.

    Information Theory, Inference and Learning Algorithms (Cambridge University Press, Cambridge, UK, 2003).

  50. 50.

    & Elements of Information Theory (John Wiley & Sons, New York, 1991).

Download references

Acknowledgements

W.J.M. is supported by award R01EY020958 from the National Eye Institute. V.N. is supported by National Science Foundation grant #0820582. J.M.B. is supported by the Gatsby Charitable Foundation and R.v.d.B. by the Netherlands Organization for Scientific Research (NWO). A.P. is supported by Multidisciplinary University Research Initiative grant N00014-07-1-0937, National Institute on Drug Abuse grant #BCS0346785, a research grant from the James S. McDonnell Foundation and award P30EY001319 from the National Eye Institute.

Author information

Author notes

    • Vidhya Navalpakkam

    Present address: Yahoo! Research, Santa Clara, California, USA.

    • Wei Ji Ma
    • , Vidhya Navalpakkam
    • , Jeffrey M Beck
    •  & Ronald van den Berg

    These authors contributed equally to this work.

Affiliations

  1. Department of Neuroscience, Baylor College of Medicine, Houston, Texas, USA.

    • Wei Ji Ma
    •  & Ronald van den Berg
  2. Department of Biology, California Institute of Technology, Pasadena, California, USA.

    • Vidhya Navalpakkam
  3. Gatsby Computational Neuroscience Unit, University College London, London, UK.

    • Jeffrey M Beck
  4. Department of Brain and Cognitive Sciences, University of Rochester, Rochester, New York, USA.

    • Alexandre Pouget

Authors

  1. Search for Wei Ji Ma in:

  2. Search for Vidhya Navalpakkam in:

  3. Search for Jeffrey M Beck in:

  4. Search for Ronald van den Berg in:

  5. Search for Alexandre Pouget in:

Contributions

W.J.M., V.N. and R.v.d.B. designed the experiments. V.N. and R.v.d.B. collected the data. W.J.M., V.N. and R.v.d.B. analyzed the data. W.J.M., J.B. and A.P. developed the theory. J.B. performed the network simulations. W.J.M. and A.P. wrote the manuscript. V.N., J.B. and R.v.d.B. contributed to the writing of the manuscript.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Wei Ji Ma.

Supplementary information

PDF files

  1. 1.

    Supplementary Text and Figures

    Supplementary Figures 1–16, Supplementary Tables 1 and 2, and Supplementary Results

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nn.2814

Further reading