Evidence accumulation during a sensorimotor decision task revealed by whole-brain imaging

Abstract

Although animals can accumulate sensory evidence over considerable time scales to appropriately select behavior, little is known about how the vertebrate brain as a whole accomplishes this. In this study, we developed a new sensorimotor decision-making assay in larval zebrafish based on whole-field visual motion. Fish responded by swimming in the direction of perceived motion, such that the latency to initiate swimming and the fraction of correct turns were modulated by motion strength. Using whole-brain functional imaging, we identified neural activity relevant to different stages of the decision-making process, including the momentary evaluation and accumulation of sensory evidence. This activity is distributed in functional clusters across different brain regions and is characterized by a wide range of time constants. In addition, we found that the caudal interpeduncular nucleus (IPN), a circular structure located ventrally on the midline of the brain, reliably encodes the left and right turning rates.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Turning behavior of larval zebrafish is dependent on motion stimulus coherence and displays accumulation of evidence.
Fig. 2: Turning behavior is dependent on both sensory and motor history.
Fig. 3: Neural correlates of the decision-making process are anatomically clustered and segregated based on stimulus and motor parameters.
Fig. 4: Fitting a general integrator model to the neural data uncovered integrator time constants and their anatomic distribution.
Fig. 5: The interpeduncular nucleus encodes the rate of left and right turns.

Data availability

Processed imaging data are available on Zenodo (https://doi.org/10.5281/zenodo.3453488). Original imaging data and behavioral data are available upon reasonable request from the corresponding author.

Code availability

Code for Figs. 4 and 5 is available on GitHub (https://github.com/portugueslab/Dragomir-et-al-2019-modelfit). Parts of the analysis can be explored online via Binder. Stimulus code is available as part of Stytra v0.8.22 (http://portugueslab.com/stytra and https://doi.org/10.5281/zenodo.3451302), and explicit experimental protocols are available on GitHub (https://github.com/portugueslab/Dragomir-et-al-2019-protocols). MATLAB code is available upon reasonable request from the corresponding author.

References

  1. 1.

    Ratcliff, R. A theory of memory retrieval. Psychol. Rev. 85, 59–108 (1978).

    Google Scholar 

  2. 2.

    Stone, M. Models for choice-reaction time. Psychometrika 25, 251–260 (1960).

    Google Scholar 

  3. 3.

    Newsome, W. T., Britten, K. H. & Movshon, J. A. Neuronal correlates of a perceptual decision. Nature 341, 52–54 (1989).

    CAS  PubMed  Google Scholar 

  4. 4.

    Shadlen, M. N. & Newsome, W. T. Motion perception: seeing and deciding. Proc. Natl Acad. Sci. USA 93, 628–633 (1996).

    CAS  PubMed  Google Scholar 

  5. 5.

    Hanes, D. P. & Schall, J. D. Neural control of voluntary movement initiation. Science 274, 427–430 (1996).

    CAS  PubMed  Google Scholar 

  6. 6.

    Gold, J. I. & Shadlen, M. N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574 (2007).

    CAS  PubMed  Google Scholar 

  7. 7.

    Scott, B. B. et al. Fronto–parietal cortical circuits encode accumulated evidence with a diversity of timescales. Neuron 95, 385–398 (2017).

    CAS  PubMed  Google Scholar 

  8. 8.

    Akrami, A., Kopec, C. D., Diamond, M. E. & Brody, C. D. Posterior parietal cortex represents sensory history and mediates its effects on behaviour. Nature 554, 368–372 (2018).

    CAS  PubMed  Google Scholar 

  9. 9.

    Licata, A. M. et al. Posterior parietal cortex guides visual decisions in rats. J. Neurosci. 37, 4954–4966 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Brunton, B. W., Botvinick, M. M. & Brody, C. D. Rats and humans can optimally accumulate evidence for decision-making. Science 340, 95–98 (2013).

    CAS  PubMed  Google Scholar 

  11. 11.

    O’Connell, R. G., Dockree, P. M. & Kelly, S. P. A supramodal accumulation-to-bound signal that determines perceptual decisions in humans. Nat. Neurosci. 15, 1729–1735 (2012).

    PubMed  Google Scholar 

  12. 12.

    Wyart, V., de Gardelle, V., Scholl, J. & Summerfield, C. Rhythmic fluctuations in evidence accumulation during decision making in the human brain. Neuron 76, 847–858 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  13. 13.

    DasGupta, S., Ferreira, C. H. & Miesenböck, G. FoxP influences the speed and accuracy of a perceptual decision in Drosophila. Science 344, 901–904 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  14. 14.

    Newsome, W. T. & Paré, E. B. A selective impairment of motion perception following lesions of the middle temporal visual area (MT). J. Neurosci. 8, 2201–2211 (1988).

    CAS  PubMed  PubMed Central  Google Scholar 

  15. 15.

    Horwitz, G. D. & Newsome, W. T. Separate signals for target selection and movement specification in the superior colliculus. Science 284, 1158–1161 (1999).

    CAS  PubMed  Google Scholar 

  16. 16.

    Kim, J. N. & Shadlen, M. N. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat. Neurosci. 2, 176–185 (1999).

    PubMed  Google Scholar 

  17. 17.

    Ding, L. & Gold, J. I. Separate, causal roles of the caudate in saccadic choice and execution in a perceptual decision task. Neuron 75, 865–874 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  18. 18.

    Hanks, T. D. et al. Distinct relationships of parietal and prefrontal cortices to evidence accumulation. Nature 520, 220–223 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  19. 19.

    Hanks, T. D., Ditterich, J. & Shadlen, M. N. Microstimulation of macaque area LIP affects decision-making in a motion discrimination task. Nat. Neurosci. 9, 682–689 (2006).

    CAS  PubMed  PubMed Central  Google Scholar 

  20. 20.

    Katz, L., Yates, J., Pillow, J. W. & Huk, A. Dissociated functional significance of choice-related activity across the primate dorsal stream. Nature 535, 285–288 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  21. 21.

    Brody, C. D. & Hanks, T. D. Neural underpinnings of the evidence accumulator. Curr. Opin. Neurobiol. 37, 149–157 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  22. 22.

    Neuhauss, S. C. et al. Genetic disorders of vision revealed by a behavioral screen of 400 essential loci in zebrafish. J. Neurosci. 19, 8603–8615 (1999).

    CAS  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Orger, M. B., Smear, M. C., Anstis, S. M. & Baier, H. Perception of Fourier and non-Fourier motion by larval zebrafish. Nat. Neurosci. 3, 1128–1133 (2000).

    CAS  PubMed  Google Scholar 

  24. 24.

    Morley, B. J. The interpeduncular nucleus. Int. Rev. Neurobiol. 28, 157–182 (1986).

    CAS  PubMed  Google Scholar 

  25. 25.

    Sharp, P. E., Turner-Williams, S. & Tuttle, S. Movement-related correlates of single cell activity in the interpeduncular nucleus and habenula of the rat during a pellet-chasing task. Behav. Brain Res. 166, 55–70 (2006).

    PubMed  Google Scholar 

  26. 26.

    Clark, B. J. & Taube, J. S. Deficits in landmark navigation and path integration after lesions of the interpeduncular nucleus. Behav. Neurosci. 123, 490–503 (2009).

    PubMed  PubMed Central  Google Scholar 

  27. 27.

    Orger, M. B., Kampff, A. R., Severi, K. E., Bollmann, J. H. & Engert, F. Control of visually guided behavior by distinct populations of spinal projection neurons. Nat. Neurosci. 11, 327–333 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  28. 28.

    Roitman, J. D. & Shadlen, M. N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002).

    CAS  PubMed  PubMed Central  Google Scholar 

  29. 29.

    Hwang, E. J., Dahlen, J. E., Mukundan, M. & Komiyama, T. History-based action selection bias in posterior parietal cortex. Nat. Commun. 8, 1242 (2017).

    PubMed  PubMed Central  Google Scholar 

  30. 30.

    Gold, J. I., Law, C.-T., Connolly, P. & Bennur, S. The relative influences of priors and sensory evidence on an oculomotor decision variable during perceptual learning. J. Neurophysiol. 100, 2653–2668 (2008).

    PubMed  PubMed Central  Google Scholar 

  31. 31.

    Portugues, R., Feierstein, C. E., Engert, F. & Orger, M. B. Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior. Neuron 81, 1328–1343 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. 32.

    Kubo, F. et al. Functional architecture of an optic flow-responsive area that drives horizontal eye movements in zebrafish. Neuron 81, 1344–1359 (2014).

    CAS  PubMed  Google Scholar 

  33. 33.

    Naumann, E. A. et al. From whole-brain data to functional circuit models: the zebrafish optomotor response. Cell 167, 947–960 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  34. 34.

    Huang, K.-H. H., Ahrens, M. B., Dunn, T. W. & Engert, F. Spinal projection neurons control turning behaviors in zebrafish. Curr. Biol. 23, 1566–1573 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  35. 35.

    Dunn, T. W. et al. Neural circuits underlying visually evoked escapes in larval zebrafish. Neuron 89, 613–628 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  36. 36.

    Wolf, S. et al. Sensorimotor computation underlying phototaxis in zebrafish. Nat. Commun. 8, 651 (2017).

    PubMed  PubMed Central  Google Scholar 

  37. 37.

    Bianco, I. H. & Wilson, S. W. The habenular nuclei: a conserved asymmetric relay station in the vertebrate brain. Philos. Trans. R. Soc. B. Biol. Sci. 364, 1005–1020 (2009).

    Google Scholar 

  38. 38.

    Bogacz, R. Optimal decision-making theories: linking neurobiology with behaviour.Trends Cogn. Sci. 11, 118–125 (2007).

    PubMed  Google Scholar 

  39. 39.

    Shadlen, M. N. & Newsome, W. T. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 86, 1916–1936 (2001).

    CAS  PubMed  Google Scholar 

  40. 40.

    Usher, M. & McClelland, J. L. The time course of perceptual choice: the leaky, competing accumulator model. Psychol. Rev. 108, 550–592 (2001).

    CAS  PubMed  Google Scholar 

  41. 41.

    Mazurek, M. E., Roitman, J. D., Ditterich, J. & Shadlen, M. N. A role for neural integrators in perceptual decision making. Cerebral Cortex 13, 1257–1269 (2003).

    PubMed  Google Scholar 

  42. 42.

    Teodorescu, A. R. & Usher, M. Disentangling decision models: from independence to competition. Psychol. Rev. 120, 1–38 (2013).

    PubMed  Google Scholar 

  43. 43.

    Portugues, R., Haesemeyer, M., Blum, M. L. & Engert, F. Whole-field visual motion drives swimming in larval zebrafish via a stochastic process. J. Exp. Biol. 218, 1433–1443 (2015).

    PubMed  PubMed Central  Google Scholar 

  44. 44.

    Severi, K. E. et al. Neural control and modulation of swimming speed in the larval zebrafish. Neuron 83, 692–707 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  45. 45.

    Robinson, D. A. Integrating with neurons. Annu. Rev. Neurosci. 12, 33–45 (1989).

    CAS  PubMed  Google Scholar 

  46. 46.

    Aksay, E., Baker, R., Seung, H. S. & Tank, D. W. Anatomy and discharge properties of pre-motor neurons in the goldfish medulla that have eye-position signals during fixations. J. Neurophysiol. 84, 1035–1049 (2000).

    CAS  PubMed  Google Scholar 

  47. 47.

    Miri, A. et al. Spatial gradients and multidimensional dynamics in a neural integrator circuit. Nat. Neurosci. 14, 1150–1161 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  48. 48.

    Daie, K., Goldman, M. S. & Aksay, E. R. F. Spatial patterns of persistent neural activity vary with the behavioral context of short-term memory. Neuron 85, 847–860 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  49. 49.

    Joshua, M. & Lisberger, S. G. A tale of two species: neural integration in zebrafish and monkeys. Neuroscience 296, 80–91 (2015).

    CAS  PubMed  Google Scholar 

  50. 50.

    Kim, D. H. et al. Pan-neuronal calcium imaging with cellular resolution in freely swimming zebrafish. Nat. Methods 14, 1107–1114 (2017).

    CAS  PubMed  Google Scholar 

  51. 51.

    Štih, V., Petrucco, L., Kist, A. M. & Portugues, R. Stytra: an open-source, integrated system for stimulation, tracking and closed-loop behavioral experiments. PLoS Comput. Biol. 15, e1006699 (2019).

    PubMed  PubMed Central  Google Scholar 

  52. 52.

    Rohlfing, T. & Maurer, C. R. Nonrigid image registration in shared-memory multiprocessor environments with application to brains, breasts, and bees. IEEE Trans. Inf. Technol. Biomed. 7, 16–25 (2003).

    PubMed  Google Scholar 

  53. 53.

    Chen, T.-W. et al. Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature 499, 295–300 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  54. 54.

    Mogensen, P. K. & Riseth, A. N. Optim: a mathematical optimization package for Julia. J. Open Source Softw. 3, 615 (2018).

    Google Scholar 

Download references

Acknowledgements

We thank M. Wulliman for advice on anatomic references. We thank A. Kist and D. Markov for technical help. We thank A. Borst and N. Gogolla for conversations and useful comments throughout the project and J. Fitzgerald for comments on the manuscript. We thank D.G.C. Hildebrand, I.H. Bianco and F. Engert (Harvard University) for sharing the Tg(elavl3:GCaMP6f) transgenic fish before publication. E.I.D. was partly funded by the Graduate School of Systemic Neurosciences, and all authors were funded by the Max Planck Gesellschaft. This research was also partially funded by Human Frontier Science Program grant RGP0027/2016 and a Life? grant from the Volkwagen Stiftung.

Author information

Affiliations

Authors

Contributions

E.I.D. and R.P. conceived the project and designed the experiments. E.I.D. performed all experiments. E.I.D. and R.P. analyzed experimental data. V.Š. and R.P. designed and implemented the model. E.I.D. and R.P. wrote the manuscript with help from V.Š.

Corresponding author

Correspondence to Ruben Portugues.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Categorization of a turn.

a, Two-dimensional contour plot showing the distance moved and angle turned averaged across all 51 fish when presented with coherence >0.5. Most swim bout events are clustered around 0 degrees, and represent forward swims, while lateralized turns begin around after 15 degrees. b, Number of left, right and forward swims as a function of stimulus coherence.

Extended Data Fig. 2 Turning response to stimulus pulses.

a, Schematic of experimental paradigm involving stimulus pulses. 6 seconds of coherence 0 were followed by a pulse coherence 0.3, 0.6 or 1. The stimulus pulse lasted 1, 2, 3, 4, 6, 8 or 10 seconds. After the pulse is over 6 seconds of coherence 0 were presented again. b, Forward swimming and turning rates for the three different coherences and the seven different pulse durations presented. The vertical grey lines denote pulse start and end. Instantaneous rates were computed by averaging over a 200 ms window. c, Average behavioral rates for forward swimming (black), turning in the direction of the stimulus (red), and against the stimulus (blue) averaged over the whole pulse as a function of the total pulse duration. In dotted lines the average baseline rates are shown computed over the 5 second window from second 1 to second 6 right before pulse onset (see panel a).

Extended Data Fig. 3 Sensory and motor history for coherences 0.3 and 1.

a, Turning behavior during trials of coherence 0.3 (top) and 1 (bottom) as a function of the coherence presented during the preceding trial. Trajectories are split by both magnitude and direction of the preceding trial. b, Latency to first correct turn in trials of coherence 0.3 (top) and 1 (bottom) as a function of the coherence of the preceding trial. c, Turning behavior during trials of coherence 0.3 (top) and 1 (bottom) as a function of both the direction of the coherence presented during the preceding trial and the direction of the last turn in the preceding trial. Only preceding trials with coherence of magnitude 0.6 were included. d, Fraction of correct first turns during trials of coherence 0.3 (top) and 1 (bottom) as a function of both the direction of the coherence of the preceding trial and the direction of the last turn in the preceding trial. Only preceding trials with coherence of magnitude 0.3 (top) and 1 (bottom) were included.

Extended Data Fig. 4 Turning behavior of head restrained larvae.

a, Example tail trace during a full head restrained experiment with the coherence stimuli sequence (positive and negative numbers represent coherences to the left and right direction, respectively). b, Average tail traces of all bouts elicited during presentation of stimuli with coherence +1 (red), 0 (black) and −1 (green) for the fish shown in a above. Leftward turns, forward swims and rightward turns, which are the predominant behaviors during these stimuli presentations, can be clearly distinguished by computing the sum of the cumulative tail angle during the first 60 ms of the bout, which we refer to as the laterality index. c, Histogram of all laterality indices for the fish in a, showing a distribution with three peaks corresponding to left- and right-ward turns and forward swims. Thresholds can be imposed to distinguish between these behaviors, in this case −7.34 separates rightward and forward swims and 2.61 separates forward swims and left-ward turns. d, Individual histograms for all bouts elicited during stimulus presentations of the different coherences for the sample fish in a. e, Left: Average fraction of correct turns (in the direction of presented coherence). Right: Average latency (time from stimulus onset) to first correct turn as a fraction of stimulus coherence. Averages over N = 18 fish; error bars denote SEM.

Extended Data Fig. 5 Brain regions sampled during imaging experiments.

Sum of projection footprints along the three axes for all brains that were imaged and could be registered to the reference brain.

Extended Data Fig. 6 Full list of regressors used in imaging experiments.

a, The regressors are constructed from the stimulus coherence shown and the tail movement recorded (both underlined) b, ROI maps color coded according to individual sensory and motor related regressors displayed in Fig. 3 (each regressor and corresponding thresholds for ROI shown here are displayed at the bottom). Each map has views from lateral left (left), dorsoventral (top central), lateral right (right) and rostro-caudal (bottom) ROI projections.

Extended Data Fig. 7 Additional model fitting of neuronal data and behavioral prediction.

a, Variance of ROIs explained by the model. The orange line is the cut of threshold for ROIs displayed in Fig. 4, chosen empirically to discard spurious fits due to artifacts. b, Simulated behavioral response of the integrator model (dashed line) to transitions in coherence and direction superimposed on data shown in Fig. 2a. The relative weights of excitation and inhibition and the nonlinearity P where extracted from data presented in Fig. 1c, whereas the time constant was extracted from behavioral data presented in Fig. 2a.

Extended Data Fig. 8 Motor-triggered neuronal activity.

a, Left: motor triggers corresponding to stepwise increases (green) and decreases (magenta) concurrent with leftward turns. The average activity of all ROIs with correlation >0.7 with the corresponding motor trigger is superimposed in black. Right: anatomical location of the motor triggers throughout the brain. Ro – rostral, c – caudal, l – left, r – right and scale bar = 300 microns. b, c, Similar to a but for neuronal activity which ramps up or down after a left turn (b) or neuronal activity which has a maximum or minimum coincident with the left turn (c). d, Motor trigger corresponding to neuronal activity that decreases steadily and is reset upon a left turn. The number of ROIs with activity that increased steadily and was reset upon a left turn was negligible. e, All ROIs with activity coincident with a forward swim.

Supplementary information

Reporting Summary

Supplementary Video 1

Freely swimming behavior of zebrafish during presentation of the RDM stimulus. The zebrafish larva is swimming in an arena consisting of an 88-mm Petri dish and being shown dark-on-white coherent dots. The larva (shown on the top right for clarity) is tracked in real time, and each stimulus is updated to appear always in the same direction with respect to its body axis. The angle of the fish’s axis is recorded (bottom right).

Supplementary Video 2

Whole-brain projection of sensory- and motor-based regression ROIs. The video shows planes from dorsal to ventral of all the ROIs identified and displayed in Fig. 3 (using the same color code, except for the motor ROIs, which are displayed here in dark blue for both directions) superimposed on the anatomy.

Supplementary Video 3

Activity in the IPN correlates with both the stimulus and directional turning behavior. Activity in the IPN averaged over three trials (rostral up). Red and green colors denote activity above and below baseline, respectively. The RDM stimulus is present on top. The trace in the bottom displays the coherence of the visual stimulus presented as a function of time in seconds (black) together with the behavior directional tail behavior (red). Positive values denote left. The acquisition rate was ~ 3 Hz. The movie is sped up eight times.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dragomir, E.I., Štih, V. & Portugues, R. Evidence accumulation during a sensorimotor decision task revealed by whole-brain imaging. Nat Neurosci 23, 85–93 (2020). https://doi.org/10.1038/s41593-019-0535-8

Download citation

Further reading

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing