Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

# Neural implementation of Bayesian inference in a sensorimotor behavior

## Abstract

Actions are guided by a Bayesian-like interaction between priors based on experience and current sensory evidence. Here we unveil a complete neural implementation of Bayesian-like behavior, including adaptation of a prior. We recorded the spiking of single neurons in the smooth eye-movement region of the frontal eye fields (FEFSEM), a region that is causally involved in smooth-pursuit eye movements. Monkeys tracked moving targets in contexts that set different priors for target speed. Before the onset of target motion, preparatory activity encodes and adapts in parallel with the behavioral adaptation of the prior. During the initiation of pursuit, FEFSEM output encodes a maximum a posteriori estimate of target speed based on a reliability-weighted combination of the prior and sensory evidence. FEFSEM responses during pursuit are sufficient both to adapt a prior that may be stored in FEFSEM and, through known downstream pathways, to cause Bayesian-like behavior in pursuit.

## Access options

from\$8.99

All prices are NET prices.

## Data and code availability

All data and custom code are available from the corresponding author upon reasonable request.

## References

1. 1.

Körding, K. P. & Wolpert, D. M. Bayesian integration in sensorimotor learning. Nature 427, 244–247 (2004).

2. 2.

Körding, K. P., Ku, S. P. & Wolpert, D. M. Bayesian integration in force estimation. J. Neurophysiol. 92, 3161–3165 (2004).

3. 3.

Körding, K. P. Bayesian statistics: relevant for the brain? Curr. Opin. Neurobiol. 25, 130–133 (2014).

4. 4.

Weiss, Y., Simoncelli, E. P. & Adelson, E. H. Motion illusions as optimal percepts. Nat. Neurosci. 5, 598–604 (2002).

5. 5.

Alais, D. & Burr, D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257–262 (2004).

6. 6.

Battaglia, P. W., Jacobs, R. A. & Aslin, R. N. Bayesian integration of visual and auditory signals for spatial localization. J. Opt. Soc. Am. A. Opt. Image. Sci. Vis. 20, 1391–1397 (2003).

7. 7.

Knill, D. C. & Saunders, J. A. Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Res. 43, 2539–2558 (2003).

8. 8.

Knill, D. C. Robust cue integration: a Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant. J. Vis. 7, 1–24 (2007).

9. 9.

Verstynen, T. & Sabes, P. N. How each movement changes the next: an experimental and theoretical study of fast adaptive priors in reaching. J. Neurosci. 31, 10050–10059 (2011).

10. 10.

Yang, J., Lee, J. & Lisberger, S. G. The interaction of Bayesian priors and sensory data and its neural circuit implementation in visually guided movement. J. Neurosci. 32, 17632–17645 (2012).

11. 11.

Darlington, T. R., Tokiyama, S. & Lisberger, S. G. Control of the strength of visual-motor transmission as the mechanism of rapid adaptation of priors for Bayesian inference in smooth pursuit eye movements. J. Neurophysiol. 118, 1173–1189 (2017).

12. 12.

Kowler, E. & Steinman, R. M. The effect of expectations on slow oculomotor control. II. Single target displacements. Vision Res. 19, 633–646 (1979b).

13. 13.

Kowler, E., Martins, A. J. & Pavel, M. The effect of expectations on slow oculomotor control--IV. Anticipatory smooth eye movements depend on prior target motions. Vision Res. 24, 197–210 (1984).

14. 14.

Kowler, E. & Steinman, R. M. The effect of expectations on slow oculomotor control. I. Periodic target steps. Vision Res. 19, 619–632 (1979a).

15. 15.

Stocker, A. A. & Simoncelli, E. P. Noise characteristics and prior expectations in human visual speed perception. Nat. Neurosci. 9, 578–585 (2006).

16. 16.

Ma, W. J. Organizing probabilistic models of perception. Trends. Cogn. Sci. 16, 511–518 (2012).

17. 17.

Ma, W. J., Beck, J. M., Latham, P. E. & Pouget, A. Bayesian inference with probabilistic population codes. Nat. Neurosci. 9, 1432–1438 (2006).

18. 18.

Lisberger, S. G. Visual guidance of smooth-pursuit eye movements: sensation, action, and what happens in between. Neuron 66, 477–491 (2010).

19. 19.

Newsome, W. T., Wurtz, R. H., Dürsteler, M. R. & Mikami, A. Deficits in visual motion processing following ibotenic acid lesions of the middle temporal visual area of the macaque monkey. J. Neurosci. 5, 825–840 (1985).

20. 20.

Krekelberg, B., van Wezel, R. J. A. & Albright, T. D. Interactions between speed and contrast tuning in the middle temporal area: implications for the neural code for speed. J. Neurosci. 26, 8988–8998 (2006).

21. 21.

Gottlieb, J. P., Bruce, C. J. & MacAvoy, M. G. Smooth eye movements elicited by microstimulation in the primate frontal eye field. J. Neurophysiol. 69, 786–799 (1993).

22. 22.

Gottlieb, J. P., MacAvoy, M. G. & Bruce, C. J. Neural responses related to smooth-pursuit eye movements and their correspondence with electrically elicited smooth eye movements in the primate frontal eye field. J. Neurophysiol. 72, 1634–1653 (1994).

23. 23.

Tanaka, M. & Lisberger, S. G. Role of arcuate frontal cortex of monkeys in smooth pursuit eye movements. I. Basic response properties to retinal image motion and position. J. Neurophysiol. 87, 2684–2699 (2002a).

24. 24.

Tanaka, M. & Lisberger, S. G. Regulation of the gain of visually guided smooth-pursuit eye movements by frontal cortex. Nature 409, 191–194 (2001).

25. 25.

Tanaka, M. & Lisberger, S. G. Enhancement of multiple components of pursuit eye movement by microstimulation in the arcuate frontal pursuit area in monkeys. J. Neurophysiol. 87, 802–818 (2002b).

26. 26.

Nuding, U. et al. TMS evidence for smooth pursuit gain control by the frontal eye fields. Cereb. Cortex 19, 1144–1150 (2009).

27. 27.

Ono, S. & Mustari, M. J. Smooth pursuit-related information processing in frontal eye field neurons that project to the NRTP. Cereb. Cortex 19, 1186–1197 (2009).

28. 28.

Brodal, P. The cortical projection to the nucleus reticularis tegmenti pontis in the rhesus monkey. Exp. Brain Res. 38, 19–27 (1980a).

29. 29.

Brodal, P. The projection from the nucleus reticularis tegmenti pontis to the cerebellum in the rhesus monkey. Exp. Brain Res. 38, 29–36 (1980b).

30. 30.

Shi, D., Friedman, H. R. & Bruce, C. J. Deficits in smooth-pursuit eye movements after muscimol inactivation within the primate’s frontal eye field. J. Neurophysiol. 80, 458–464 (1998).

31. 31.

Keating, E. G. Frontal eye field lesions impair predictive and visually-guided pursuit eye movements. Exp. Brain Res. 86, 311–323 (1991).

32. 32.

MacAvoy, M. G., Gottlieb, J. P. & Bruce, C. J. Smooth-pursuit eye movement representation in the primate frontal eye field. Cereb. Cortex 1, 95–102 (1991).

33. 33.

Lee, J., Yang, J. & Lisberger, S. G. Control of the gain of visual-motor transmission occurs in visual coordinates for smooth pursuit eye movements. J. Neurosci. 33, 9420–9430 (2013).

34. 34.

Tanaka, M. & Fukushima, K. Neuronal responses related to smooth pursuit eye movements in the periarcuate cortical area of monkeys. J. Neurophysiol. 80, 28–47 (1998).

35. 35.

Mahaffy, S. & Krauzlis, R. J. Neural activity in the frontal pursuit area does not underlie pursuit target selection. Vision Res. 51, 853–866 (2011).

36. 36.

Snyder, A. C., Morais, M. J. & Smith, M. A. Dynamics of excitatory and inhibitory networks are differentially altered by selective attention. J. Neurophysiol. 116, 1807–1820 (2016).

37. 37.

Lim, S. & Goldman, M. S. Balanced cortical microcircuitry for maintaining information in working memory. Nat. Neurosci. 16, 1306–1314 (2013).

38. 38.

Beck, J. M. et al. Probabilistic population codes for Bayesian decision making. Neuron 60, 1142–1152 (2008).

39. 39.

Maunsell, J. H. & Van Essen, D. C. Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. J. Neurophysiol. 49, 1127–1147 (1983).

40. 40.

Mikami, A., Newsome, W. T. & Wurtz, R. H. Motion selectivity in macaque visual cortex. I. Mechanisms of direction and speed selectivity in extrastriate area MT. J. Neurophysiol. 55, 1308–1327 (1986).

41. 41.

Salinas, E. & Abbott, L. F. Vector reconstruction from firing rates. J. Comput. Neurosci. 1, 89–107 (1994).

42. 42.

Groh, J. M. Converting neural signals from place codes to rate codes. Biol. Cybern. 85, 159–165 (2001).

43. 43.

Churchland, M. M. & Lisberger, S. G. Shifts in the population response in the middle temporal visual area parallel perceptual and motor illusions produced by apparent motion. J. Neurosci. 21, 9387–9402 (2001).

44. 44.

Bogadhi, A. R., Montagnini, A. & Masson, G. S. Dynamic interaction between retinal and extraretinal signals in motion integration for smooth pursuit. J. Vis. 13, 5 (2013).

45. 45.

Schwartz, J. D. & Lisberger, S. G. Initial tracking conditions modulate the gain of visuo-motor transmission for smooth pursuit eye movements in monkeys. Vis. Neurosci. 11, 411–424 (1994).

46. 46.

Fuchs, A. F. & Robinson, D. A. A method for measuring horizontal and vertical eye movement chronically in the monkey. J. Appl. Physiol. 21, 1068–1070 (1966).

47. 47.

Ramachandran, R. & Lisberger, S. G. Normal performance and expression of learning in the vestibulo-ocular reflex (VOR) at high frequencies. J. Neurophysiol. 93, 2028–2038 (2005).

48. 48.

Rashbass, C. The relationship between saccadic and smooth tracking eye movements. J. Physiol. (Lond.) 159, 326–338 (1961).

49. 49.

Osborne, L. C., Hohl, S. S., Bialek, W. & Lisberger, S. G. Time course of precision in smooth-pursuit eye movements of monkeys. J. Neurosci. 27, 2987–2998 (2007).

50. 50.

Lisberger, S. G. & Westbrook, L. E. Properties of visual inputs that initiate horizontal smooth pursuit eye movements in monkeys. J. Neurosci. 5, 1662–1673 (1985).

## Acknowledgements

We thank S. Tokiyama and S. Happel for technical assistance, G. Field for comments on an earlier version of the manuscript, and Mhrdad Jazayeri for encouragement. We are grateful to N. Brunel for helpful guidance in the network modeling and constructive feedback on the paper and to A. Rosko for pointing out a useful plasticity rule. NIH R01-EY027373 (S.G.L.) and a gift from the Howard Hughes Medical Institute supported the research. T.R.D. received support from the Wakeman Endowment Fund, the Duke University Medical Scientist Training Program (T32 GM007171), and from NIH award F30-EY027684 (T.R.D.).

## Author information

Authors

### Contributions

Conceptualization: T.R.D. and S.G.L.; methodology: T.R.D. and S.G.L.; formal analysis: T.R.D., J.M.B., and S.G.L.; investigation: T.R.D.; writing (original draft): T.R.D.; writing (review and editing): T.R.D., J.M.B., and S.G.L.; supervision: S.G.L.; funding acquisition: S.G.L.

### Corresponding author

Correspondence to Stephen G. Lisberger.

## Ethics declarations

### Competing interests

The authors declare no competing interests.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Integrated supplementary information

### Supplementary Figure 1 A Bayesian model for the effects of context on speed estimation.

Using the approach of Stocker and Simoncelli15 (Nat Neurosci. 9: 578–585, 2006), this analysis shows that pursuit behavior is Bayes optimal. Here, the posterior distributions were estimated as the distributions of single-trial eye velocities during the initiation of pursuit. We can model the distributions of the initial eye velocity of pursuit using Bayes’ law if we constrain the probability distributions to be Gaussian and we assume that the likelihood distribution peaks at actual target speed, 10 deg/s. The best fitting model has a lower amplitude and wider likelihood distribution for weak versus strong visual motion, and uses a prior distribution that peaks at a higher target speed for the fast- versus the slow-context. (a–d): Solid curves plot distributions of eye speed in response to high-contrast (red) and low-contrast (black) targets moving at 10 deg/s during the fast-context (a, b) and slow-context (c, d). Dashed curves plot posterior distributions over target speed resulting from the Bayesian fit. The fits in (a–d) allowed the variance of the prior distributions to vary independently, but were just as good if we constrained the variance of the prior distributions to be equal across contexts. (e, f): Red and black dashed curves plot the likelihood distributions over target speed for the high- and low-contrast targets generated from the Bayesian fit. Green and blue solid curves plot prior distributions for the fast- and slow-contexts generated from the Bayesian fit. This figure documents one additional important feature of Bayesian behavior on pursuit eye movements. In contrast to the bimodal distributions found for human perceptual decisions in a recent paper (Laquitaine S. and Gardner J.L. A switching observer for human perceptual estimation. Neuron 97: 1–13, 2018), the distributions in panels (a–d) are unimodal based on measurements from individual trials, confirming that our system follows the expectations for Bayesian inference. The priors derived for our model shift toward faster speeds and slightly flatten out in the fast- compared to the slow-context. The model predicts wider likelihood functions for the low- compared to the high-contrast target, consistent with the notion that the low-contrast target creates less-reliable visual motion signals in the brain. Thus, our behavioral data demonstrate a reliability-weighted combination of priors and sensory evidence. Figure is reprinted from Figure 5 in Darlington et al.11 (J. Neurophysiol. 118: 1173–89, 2017).

### Supplementary Figure 2 Temporal evolution of preparatory activity and behavioral effects across the fast and slow contexts.

The contextual effects on preparatory firing rate and eye speed evolved similarly across both the fast and slow context. Whereas Fig. 3a in the main paper compares the difference between the fast and slow contexts in eye speed and preparatory activity, here we consider the time course of adaptation in each context independently. Preparatory activity and eye speed in response to the 20 deg/s (a) and 2 deg/s (b) target motion across each 50-trial block were normalized to the first 10 trials. Open symbols in (a) and (b) represent normalized eye speed averaged across experiments and filled symbols represent the normalized population preparatory activity. Red and black open symbols show behavioral data for high-contrast versus low-contrast targets. Error bars represent SEM (n = 95 speed context experiments). Preparatory activity and eye speeds increased across the fast context and decreased across the slow context. In both contexts, the effects of context on eye speed for the low-contrast target nearly matched the effects on preparatory activity. As expected in a Bayesian framework, the effects of contrast were smaller for the high-contrast target.

### Supplementary Figure 3 Results do not depend on stimulus form.

Our experiments used different combinations of stimuli that consisted of patches of dots and Gabor functions, that is sine wave gratings vignetted within a two-dimensional Gaussian window. For the figures in the main paper, we mixed experiments where the high- and low-contrast targets had different forms with experiments where they were of the same form. Here, we show that the results are the same whether the stimulus forms were the same or mixed. The filled symbols in (a), (b), and (c) represent data collected in experiments using matched-stimulus form and the open symbols in (d), (e), and (f) represent data collected in experiments using mixed-stimulus form. The bottom line is that the effects are essentially the same in the top and bottom rows of graphs. (a, d): Scatter plots compare normalized eye speed in response to high- versus low-contrast targets for 10 deg/s target motion during the fast (green) and slow (blue) context. (b, e): Scatter plots compare the average change in preparatory firing rate across fixation during the fast versus slow context. The solid and open lines were obtained via regression and the dashed lines represent unity. (c, f): Scatter plots compare the effect of speed context on pursuit-related firing rate for the high- and low-contrast targets. Red and black marginal histograms show the distributions of speed context effects on firing during the initiation of pursuit for the high- and low-contrast targets. There are no statistically significant differences between data collected in experiments that matched versus mixed stimulus forms. For (a) versus (d), fast-context, low-contrast: meanmatched (±S.E.M.) = 12.08 ± 1.51 %, meanmixed (±S.E.M.) = 12.24 ± 0.86 %, p = 0.92 (two-sample, two-tailed t-test), t109 = −0.10 (nmatch = 36 behavioral experiments, nmix = 75 behavioral experiments). For (a) versus (d), fast-context, high-contrast: meanmatched (±S.E.M.) = 7.18 ± 0.91 %, meanmixed (±S.E.M.) = 7.00 ± 0.67 %, p = 0.88 (two-sample, two-tailed t-test), t109 = 0.15 (nmatch = 36 behavioral experiments, nmix = 75 behavioral experiments). For (a) versus (d), slow-context, low-contrast: meanmatched (±S.E.M.) = −19.36 ± 1.63%, meanmixed (±S.E.M.) = 16.92 ± 0.75 %, p = 0.12 (two-sample, two-tailed t-test), t109 = −1.56 (nmatch = 36 behavioral experiments, nmix = 75 behavioral experiments). For (a) versus (d), slow-context, high-contrast: meanmatched (±S.E.M.) = −11.99 ± 0.93%, meanmixed (±S.E.M.) = −10.02 ± 0.68 %, p = 0.10 (two-sample, two-tailed t-test), t109 = −1.68 (nmatch = 36 behavioral experiments, nmix = 75 behavioral experiments). For (b) versus (e), fast-context: meanmatched (±S.E.M.) = 5.86 ± 1.61 spikes/s, meanmixed (±S.E.M.) = 7.57 ± 0.99 spikes/s, p = 0.37 (two-sample, two-tailed t-test), t319 = −0.89 (nmatch = 83 data points from 49 cells, nmix = 238 data points from 115 cells). For (b) versus (e), slow-context: meanmatched (±S.E.M.) = 3.77 ± 1.22 spikes/s, meanmixed (±S.E.M.) = 6.35 ± 0.78 spikes/s, p = 0.09 (two-sample, two-tailed t-test), t319 = −1.72 (nmatch = 83 data points from 49 cells, nmix = 238 data points from 115 cells). For (c) versus (f), high-contrast: meanmatched (±S.E.M.) = 2.34 ± 2.44 spikes/s, meanmixed = −0.56 ± 1.52 spikes/s, p = 0.36 (two-sample, two-tailed t-test), t162 = 0.92 (nmatched = 36 data points from 27 cells, nmixed = 128 data points from 91 cells). For (c) versus (f), low-contrast: meanmatched (±S.E.M.) = 8.20 ± 2.99 spikes/s, meanmixed = 5.68 ± 2.20 spikes/s, p = 0.57 (two-sample, two-tailed t-test), t162 = 0.57 (nmatched = 36 data points from 27 cells, nmixed = 128 data points from 91 cells).

### Supplementary Figure 4 Parameter sensitivity of a model described by Equation (3) that updates preparatory activity based on previous pursuit responses.

(a–c): Symbols plot the time course of the ratio of preparatory activity between fast and slow contexts for the model. The black symbols in each panel show the performance of the optimized model presented in Fig. 6 for w1 = 1.93, w2 = −1.26 and trial memory = 30 trials. In (a), the value of w2 was constant and different colors show the effect of adjusting w1. In (b), w1 was constant and different colors show the effect of adjusting w2. In (c), different colors show the effect of using different amounts of trial memory. (d, e): Solid curves represent distributions of normalized pursuit-related firing rate (as in Fig. 6a). Dashed and dotted curves show distributions of normalized pursuit-related firing rate that have been shifted to the right by 0.125 or 0.25 units. The distributions for target motion at 2 deg/s were shifted in (d) and those for target motion at 20 deg/s in (e. f): Symbols plot the trial course of the relative change in preparatory activity between fast and slow contexts for the model. Blue and green symbols plot the result when the 2 and 20 deg/s distributions were right-shifted by 0.125 (connected by dashed line) or 0.25 (connected by dotted line). Black symbols plot the result when the actual distributions from our recordings were used. If we shift the 2 deg/s curves to the right, then the state variable plateaus at a smaller difference between the fast- and slow-context (blue traces). If we shift the 20 deg/s curves to the right, then the state variable plateaus at a larger difference between the fast- and slow-context (green traces). Comparison across all panels shows that the level of the plateau in the difference between contexts is controlled only by the relative differences in pursuit-related firing rates for target motions at 20, 10, and 2 deg/s. Therefore, the plateau of the state variable emerges seamlessly from the measured distributions of pursuit firing rates in FEFSEM.

### Supplementary Figure 5 Model MT population responses for motion of low-contrast and high-contrast targets at 2°/s, 10°/s, and 20°/s.

The two plots show data for the motion of targets of low-contrast (12%-contrast patches of dots) and high-contrast (100%-contrast patches of dots). Each symbol shows the responses of a different model neuron and plots that neuron’s firing rate as a function of preferred speed. Open red circles, green dots, and blue x’es show responses for target motion at 2, 10, and 20 deg/s. Plots were constructed by drawing 1/3 of the 1026 model neurons in the full sample used in the model. The model neurons were obtained by extrapolation of the responses of 62 MT neurons studied with multiple target speeds and stimulus contrasts (Jin Yang and Stephen Lisberger, unpublished data). We first identified a subset of our sample of actual neurons that were tuned for speed, by eliminating the small minority that were low-pass or high-pass, so that the remaining tuning curves could be shifted horizontally along the preferred speed axis without creating bogus model responses. We created the 1026 model neurons to have preferred speeds that were uniformly distributed on a log-speed scale from 0.27 to 127. For each model neuron, we randomly chose a real neuron, shifted the real neuron’s tuning curve to have the preferred speed of the specified model neuron, and then computed the model neuron’s responses for target speeds of 2, 10, and 20 deg/s for low-contrast and high-contrast targets. The arrows in each figure show the estimates of target speed for each the population responses, calculated as the center of mass on a log2 axis: $$speed = 2^{\left( {\frac{{\mathop {\sum}\nolimits_i {MT_i} \cdot log_2PS_i}}{{\mathop {\sum}\nolimits_i {MT_i} }}} \right)}$$, where MTi and PSi are the response amplitude and preferred speed of the ith model MT neuron. Inspection of the two graphs in this figure show the two important features of the model MT population responses. (1) The peak of each population responses is at or near the speed of the target motion, but is slightly higher for low- versus high-contrast targets, as pointed out by Krekelberg et al.20 (J. Neurosci. 26: 8988–8998, 2006). (2) The amplitude of the population responses is higher for the high-contrast than the low-contrast targets, with many weak or absent responses for the low-contrast targets.

## Supplementary information

### Supplementary Text and Figures

Supplementary Figures 1–5

## Rights and permissions

Reprints and Permissions

Darlington, T.R., Beck, J.M. & Lisberger, S.G. Neural implementation of Bayesian inference in a sensorimotor behavior. Nat Neurosci 21, 1442–1451 (2018). https://doi.org/10.1038/s41593-018-0233-y

• Accepted:

• Published:

• Issue Date:

• ### Optimality and heuristics in perceptual neuroscience

• Justin L. Gardner

Nature Neuroscience (2019)