During perceptual decision-making, responses in the middle temporal (MT) and lateral intraparietal (LIP) areas appear to map onto theoretically defined quantities, with MT representing instantaneous motion evidence and LIP reflecting the accumulated evidence. However, several aspects of the transformation between the two areas have not been empirically tested. We therefore performed multistage systems identification analyses of the simultaneous activity of MT and LIP during individual decisions. We found that monkeys based their choices on evidence presented in early epochs of the motion stimulus and that substantial early weighting of motion was present in MT responses. LIP responses recapitulated MT early weighting and contained a choice-dependent buildup that was distinguishable from motion integration. Furthermore, trial-by-trial variability in LIP did not depend on MT activity. These results identify important deviations from idealizations of MT and LIP and motivate inquiry into sensorimotor computations that may intervene between MT and LIP.
Subscribe to Journal
Get full journal access for 1 year
only $18.75 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Newsome, W.T. & Paré, E.B. A selective impairment of motion perception following lesions of the middle temporal visual area (MT). J. Neurosci. 8, 2201–2211 (1988).
Salzman, C.D., Britten, K.H. & Newsome, W.T. Cortical microstimulation influences perceptual judgements of motion direction. Nature 346, 174–177 (1990).
Roitman, J.D. & Shadlen, M.N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002).
Gold, J.I. & Shadlen, M.N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574 (2007).
Shadlen, M.N. & Newsome, W.T. Motion perception: seeing and deciding. Proc. Natl. Acad. Sci. USA 93, 628–633 (1996).
Mazurek, M.E., Roitman, J.D., Ditterich, J. & Shadlen, M.N. A role for neural integrators in perceptual decision making. Cereb. Cortex 13, 1257–1269 (2003).
Beck, J.M. et al. Probabilistic population codes for Bayesian decision making. Neuron 60, 1142–1152 (2008).
Wimmer, K. et al. Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT. Nat. Commun. 6, 6177 (2015).
Cicmil, N., Cumming, B.G., Parker, A.J. & Krug, K. Reward modulates the effect of visual cortical microstimulation on perceptual decisions. Elife 4, e07832 (2015).
Law, C.-T. & Gold, J.I. Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area. Nat. Neurosci. 11, 505–513 (2008).
Kiani, R., Hanks, T.D. & Shadlen, M.N. Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment. J. Neurosci. 28, 3017–3029 (2008).
Rao, V., DeAngelis, G.C. & Snyder, L.H. Neural correlates of prior expectations of motion in the lateral intraparietal and middle temporal areas. J. Neurosci. 32, 10063–10074 (2012).
Meister, M.L.R., Hennig, J.A. & Huk, A.C. Signal multiplexing and single-neuron computations in lateral intraparietal area during decision-making. J. Neurosci. 33, 2254–2267 (2013).
Park, I.M., Meister, M.L.R., Huk, A.C. & Pillow, J.W. Encoding and decoding in parietal cortex during sensorimotor decision-making. Nat. Neurosci. 17, 1395–1403 (2014).
Churchland, A.K., Kiani, R. & Shadlen, M.N. Decision-making with multiple alternatives. Nat. Neurosci. 11, 693–702 (2008).
Hanks, T.D., Mazurek, M.E., Kiani, R., Hopp, E. & Shadlen, M.N. Elapsed decision time affects the weighting of prior probability in a perceptual decision task. J. Neurosci. 31, 6339–6352 (2011).
Louie, K. & Glimcher, P.W. Separating value from choice: delay discounting activity in the lateral intraparietal area. J. Neurosci. 30, 5498–5507 (2010).
Bennur, S. & Gold, J.I. Distinct representations of a perceptual decision and the associated oculomotor plan in the monkey lateral intraparietal area. J. Neurosci. 31, 913–921 (2011).
Rishel, C.A., Huang, G. & Freedman, D.J. Independent category and spatial encoding in parietal cortex. Neuron 77, 969–979 (2013).
Jazayeri, M. & Shadlen, M.N. A neural mechanism for sensing and reproducing a time interval. Curr. Biol. 25, 2599–2609 (2015).
Huk, A.C. & Shadlen, M.N. Neural activity in macaque parietal cortex reflects temporal integration of visual motion signals during perceptual decision making. J. Neurosci. 25, 10420–10436 (2005).
Katz, L.N., Yates, J.L., Pillow, J.W. & Huk, A.C. Dissociated functional significance of decision-related activity in the primate dorsal stream. Nature 535, 285–288 (2016).
de Lafuente, V., Jazayeri, M. & Shadlen, M.N. Representation of accumulating evidence for a decision in two parietal areas. J. Neurosci. 35, 4306–4318 (2015).
Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. Responses of neurons in macaque MT to stochastic motion signals. Vis. Neurosci. 10, 1157–1169 (1993).
Britten, K.H., Newsome, W.T., Shadlen, M.N., Celebrini, S. & Movshon, J.A. A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis. Neurosci. 13, 87–100 (1996).
Pillow, J.W. et al. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454, 995–999 (2008).
Mineault, P.J., Tring, E., Trachtenberg, J.T. & Ringach, D.L. Enhanced spatial resolution during locomotion and heightened attention in mouse primary visual cortex. J. Neurosci. 36, 6382–6392 (2016).
Gnadt, J.W. & Andersen, R.A. Memory related motor planning activity in posterior parietal cortex of macaque. Exp. Brain Res. 70, 216–220 (1988).
Wurtz, R.H., Sommer, M.A., Paré, M. & Ferraina, S. Signal transformations from cerebral cortex to superior colliculus for the generation of saccades. Vision Res. 41, 3399–3412 (2001).
Nienborg, H. & Cumming, B.G. Decision-related activity in sensory neurons reflects more than a neuron's causal effect. Nature 459, 89–92 (2009).
Brunton, B.W., Botvinick, M.M. & Brody, C.D. Rats and humans can optimally accumulate evidence for decision-making. Science 340, 95–98 (2013).
Raposo, D., Kaufman, M.T. & Churchland, A.K. A category-free neural population supports evolving demands during decision-making. Nat. Neurosci. 17, 1784–1792 (2014).
Wyart, V., de Gardelle, V., Scholl, J. & Summerfield, C. Rhythmic fluctuations in evidence accumulation during decision making in the human brain. Neuron 76, 847–858 (2012).
Mazzoni, P., Bracewell, R.M., Barash, S. & Andersen, R.A. Motor intention activity in the macaque's lateral intraparietal area. I. Dissociation of motor plan from sensory memory. J. Neurophysiol. 76, 1439–1456 (1996).
Janssen, P. & Shadlen, M.N. A representation of the hazard rate of elapsed time in macaque area LIP. Nat. Neurosci. 8, 234–241 (2005).
Ungerleider, L.G. & Desimone, R. Cortical connections of visual area MT in the macaque. J. Comp. Neurol. 248, 190–222 (1986).
Tauste Campo, A. et al. Task-driven intra- and interarea communications in primate cerebral cortex. Proc. Natl. Acad. Sci. USA 112, 4761–4766 (2015).
Ruff, D.A. & Cohen, M.R. Attention increases spike count correlations between visual cortical areas. J. Neurosci. 36, 7523–7534 (2016).
Cisek, P., Puskas, G.A. & El-Murr, S. Decisions in changing conditions: the urgency-gating model. J. Neurosci. 29, 11560–11571 (2009).
Thura, D., Beauregard-Racine, J., Fradet, C.W. & Cisek, P. Decision making by urgency gating: theory and experimental support. J. Neurophysiol. 108, 2912–2930 (2012).
Ditterich, J. Stochastic models of decisions about motion direction: behavior and physiology. Neural Netw. 19, 981–1012 (2006).
Latimer, K.W., Yates, J.L., Meister, M.L.R., Huk, A.C. & Pillow, J.W. Neuronal modeling. Single-trial spike trains in parietal cortex reveal discrete steps during decision-making. Science 349, 184–187 (2015).
Scott, B.B., Constantinople, C.M., Erlich, J.C., Tank, D.W. & Brody, C.D. Sources of noise during accumulation of evidence in unrestrained and voluntarily head-restrained rats. Elife 4, e11308 (2015).
Hanks, T.D. et al. Distinct relationships of parietal and prefrontal cortices to evidence accumulation. Nature 520, 220–223 (2015).
Carland, M.A., Marcos, E., Thura, D. & Cisek, P. Evidence against perfect integration of sensory information during perceptual decision making. J. Neurophysiol. 115, 915–930 (2016).
Joo, S.J., Katz, L.N. & Huk, A.C. Decision-related perturbations of decision-irrelevant eye movements. Proc. Natl. Acad. Sci. USA 113, 1925–1930 (2016).
Ding, L. Distinct dynamics of ramping activity in the frontal cortex and caudate nucleus in monkeys. J. Neurophysiol. 114, 1850–1861 (2015).
Komura, Y., Nikkuni, A., Hirashima, N., Uetake, T. & Miyamoto, A. Responses of pulvinar neurons reflect a subject's confidence in visual categorization. Nat. Neurosci. 16, 749–755 (2013).
Brainard, D.H. The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997).
Kleiner, M. et al. What's new in Psychtoolbox-3. Perception 36, 1 (2007).
Eastman, K.M. & Huk, A.C. Pldaps: a hardware architecture and software toolbox for neurophysiology requiring complex visual stimuli and online behavioral control. Front. Neuroinform. 6, 1 (2012).
Albright, T.D. Direction and orientation selectivity of neurons in visual area MT of the macaque. J. Neurophysiol. 52, 1106–1130 (1984).
Bair, W. & Movshon, J.A. Adaptive temporal integration of motion in direction-selective neurons in macaque visual cortex. J. Neurosci. 24, 7305–7323 (2004).
Van Essen, D.C., Newsome, W.T. & Maunsell, J.H. The visual field representation in striate cortex of the macaque monkey: asymmetries, anisotropies, and individual variability. Vision Res. 24, 429–448 (1984).
Mineault, P.J., Khawaja, F.A., Butts, D.A. & Pack, C.C. Hierarchical processing of complex motion along the primate dorsal visual pathway. Proc. Natl. Acad. Sci. USA 109, E972–E980 (2012).
Cui, Y., Liu, L.D., Khawaja, F.A., Pack, C.C. & Butts, D.A. Diverse suppressive influences in area MT and selectivity to complex motion features. J. Neurosci. 33, 16715–16728 (2013).
Truccolo, W., Eden, U.T., Fellows, M.R., Donoghue, J.P. & Brown, E.N. A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. J. Neurophysiol. 93, 1074–1089 (2005).
Pillow, J.W., Paninski, L., Uzzell, V.J., Simoncelli, E.P. & Chichilnisky, E.J. Prediction and decoding of retinal ganglion cell responses with a probabilistic spiking model. J. Neurosci. 25, 11003–11013 (2005).
This research was supported by a Howard Hughes Medical Institute International Student Research Fellowship to L.N.K., a McKnight Foundation grant to J.W.P., a National Eye Institute (R01-EY017366) grant to both J.W.P. and A.C.H., and National Institutes of Health under Ruth L. Kirschstein National Research Service Awards T32DA018926 from the National Institute on Drug Abuse and T32EY021462 from the National Eye Institute.
The authors declare no competing financial interests.
Integrated supplementary information
(a) Psychophysical performance for monkey P (10,280 trials). Motion strengths from all trials were binned into 30 evenly-spaced quantiles. Error bars indicate ±1 s.e.m. Solid lines indicate fits from a 4-parameter logistic function. (b) Psychophysical performance for monkey N (12,558 trials) using the same analysis and binning as in (a). (c) Psychophysical kernel for monkey P, measured using logistic regression. Error bars indicate ±1 s.e.m. (d) Psychophysical kernel for monkey N.
Our task involved a novel manipulation of motion strength compared with previous studies using random-dot stimuli. As such, we obtained a point of reference to other studies by binning and plotting our data using similar smoothing, normalization, bin sizes, and aspect ratios. Comparison to the existing literature shows that our LIP data exhibit qualitatively similar ramping average responses to previous studies – and that the parameters guiding normalization and smoothing can have important effects on the appearance of ramping and coherence dependence in plotted responses. (a) Our LIP data plotted using the same binning, smoothing and aspect ratio as Shadlen and Newsome (2001): Here, responses were binned at a 50ms resolution and averaged without normalization (n=104). Comparing this plot to figure 8 from Shadlen in Newsome (2001) shows that, on average, our population has similar baseline firing rates and ramping average dynamics. (b) Our LIP data plotted using the same binning, smoothing, normalization, and aspect ratio as Kiani and Shadlen (2009). Here, we normalized by mean activity before motion onset, smoothed with a 100ms boxcar filter, binned in 10ms bins. This visualization of our data closely resembles supplemental figure S2 in Kiani and Shadlen (2009). (c) Our LIP data plotted using the same binning, smoothing, normalization, and aspect ratio as ref. 24. The population average PSTH was normalized by delay period activity, smoothed by an exponential kernel (τ=30ms) and binned in 10ms bins. Our data closely resembles figure 8 from ref. 4.
Prior work examining LIP’s role in representing the temporal integration of evidence looked at the effect of single motion pulses on LIP firing rates after conditioning on each choice separately22. In the main text, we computed the raw PTA without conditioning on the choice for each trial. Here, we examine the effects of motion pulses after correcting for the choice. We also examine the effects of the choice-stimulus correlation by calculating the PTA for after replacing the spike rate on each trial with the choice-specific PSTH. (a) PTA corrected for choice on each trial. This analysis calculates the PTA of the residual spike counts after subtracting off the choice-specific PSTH for each trial. This analysis demonstrates that after correcting for choice, motion has a decreasing effect on LIP firing rate throughout the trial. (b) The effect of choice on the PTA. To calculate the effect of choice on the PTA, we replaced each trial with the corresponding trial’s choice-conditioned average firing rate and then calculated the PTA for those firing rates. This demonstrates that if LIP were only correlated with the choice (i.e., had no additional dependence on motion), we would identify a systematic correlation between each pulse and LIP firing rate with ramping average dynamics.
This LIP unit had a significant improvement from the inclusion of MT coupling. However, it exhibited unconventional response dynamics, and weak if any selectivity. (a) Choice and motion-strength sorted PSTH. Motion strengths were divided into quartiles for each choice. Spikes were binned at a 10ms resolution and smoothed with a 200ms boxcar. (b) Log-likelihood ratio per trial (LLR) for the coupled model compared to the uncoupled model as a function of time. Gray traces are individual cross-validation folds. Yellow is the average across folds in a sliding 100ms window. (c) PSTHs for simultaneously recorded MT units (same analysis as in a). (d) The LIP unit’s coupling filters for corresponding MT neurons. Time is lags relative to the MT spike time. Gray traces are cross-validation folds. Yellow is the average across folds. (e) Shuffle-corrected cross correlograms (CCGs) for MT-LIP pairs. (Top row) CCGs for MT-LIP pairs during motion epoch only in 1ms bins (gray) and smoothed by a 10ms boxcar (black) compared to a reference line at zero (cyan). Shuffle correction was performed for all stimulus conditions so these CCGs include signal and noise correlations, but have been corrected for large temporal fluctuations in the mean rates. Even without correcting for the drive from pulses, any correlation between these units during the motion epoch is minimal. (Bottom row) CCGs using all spikes that occurred within the trial. The strong peak in the CCG in the left shows the LIP unit firing before an MT unit. This correlation is likely driven by the timing of saccadic eye-movements that were not controlled for in these plots. No saccades are present in the CCGs from the motion epoch (top row).
(a) Choice and motion sorted PSTH (same analysis as Supplementary Figure 4a). (b) Log-likelihood ratio per trial (LLR) comparing the coupled MT-LIP model to the uncoupled model. Gray traces are individual cross-validation folds. Yellow is the average across folds in a sliding 100ms window. The coupled model was more likely than the uncoupled model throughout the trial. (c) PSTHs for simultaneously recorded MT units (same analysis as in a). The middle unit is not directionally selective and has very unconventional responses during the motion epoch. It is likely that the motion stimulus was not in the receptive field. (d) The LIP unit’s coupling filters for the corresponding MT units in c. These filters show a strong coupling at short time-lags to the second MT unit (y-axis is scaled differently for the three plots) (e) Shuffle-corrected cross-correlograms (CCGs) for each MT-LIP pair. (Top row) CCGs during the motion epoch. Like the coupling filters, the CCGs suggest strong inter-areal coupling between the second MT unit and the LIP unit at short time lags. (Bottom row) CCGs using all spikes that occurred throughout the whole trial epoch show the same strong correlation between the second MT unit and the LIP unit at short time lags. The unconventional response properties of this MT unit make it difficult to interpret the functional significance of this coupling in terms of an integrator model.
(a) Choice and motion-strength sorted PSTH (same analysis as in Supplementary Figures 4, 5). This LIP unit exhibited unconventional responses during the decision epoch and was driven most strongly by the onset of the motion stimulus. (b) Log-likelihood ratio per trial (LLR) for the coupled model compared to the uncoupled model as a function of time (aligned to motion onset). (c) PSTHs for simultaneously recorded MT units (same analysis as a). These units had conventional responses to motion. (d) The LIP units’ coupling filters for the corresponding MT units in c. Gray indicates individual cross-validation folds. Yellow is the average across folds. The coupling filters show that the LIP unit was slightly suppressed by activity in the third MT unit over long timescales. (e) Shuffle-corrected cross-correlograms (CCGs) for MT-LIP pairs. (Top row) CCGs for MT-LIP pairs during the motion epoch. The CCGs show that the LIP unit fired less relative to spikes before and after spikes from the third MT unit over long time-lags. (Bottom row) CCGs using all spikes that occurred within the trial show the same correlation structure as during motion. The correlations for these units are symmetric and slow-timescale. Thus, they are difficult to interpret within the framework of feedforward integration.
Previous work has reported ramping activity in LIP that was not linked to the integration of motion, termed an urgency signal16. We calculated the urgency signal in our data by averaging all response to the choice-unconditioned zero-expected-motion trials. This analysis confirmed that the urgency signal does not play a large role in this dataset. (a) Urgency for monkey P (n=40). (b) Urgency signal for monkey N (n=64).
(a) PSTH for all LIP units with RFs overlapping the Gabor stimulus that showed strong visual onset transients (n=30). The data (points) are overlaid with the MT-to-LIP model prediction (lines). Motion strength and choice conditioning is the same as in the main text (e.g., Figure 2). (b) PSTH for the subset of LIP units from a that had simultaneously recorded MT data (n=13). (c) Choice kernels for choices in the preferred direction of the unit (blue) and anti-preferred direction (red) aligned to saccade onset. Different color strengths correspond to the different truncations in panel e. The choice kernels for LIP units with motion in their RF were small compared to the LIP units with the target in their RF (Figure 5). (d) MT-LIP coupling filters for data (yellow) and independent surrogates (gray) shows that the observed coupling was not outside the null range. (e) Variance of the PSTH explained as a function of the length of the choice kernel. Error bars indicate ±1 s.e.m. Including more time before the saccade modestly improved the fit quality for LIP units with the motion in their RF. (f) Model comparison for the GLM with MT-LIP coupling to an uncoupled model. There are no systematic differences from the distribution in Figure 6d of the main text.
About this article
Cite this article
Yates, J., Park, I., Katz, L. et al. Functional dissection of signal and noise in MT and LIP during decision-making. Nat Neurosci 20, 1285–1292 (2017). https://doi.org/10.1038/nn.4611
Annals of the New York Academy of Sciences (2020)
PLOS Computational Biology (2020)
Current Opinion in Physiology (2020)
Current Opinion in Neurobiology (2020)