Abstract
Prefrontal cortex is thought to have a fundamental role in flexible, contextdependent behaviour, but the exact nature of the computations underlying this role remains largely unknown. In particular, individual prefrontal neurons often generate remarkably complex responses that defy deep understanding of their contribution to behaviour. Here we study prefrontal cortex activity in macaque monkeys trained to flexibly select and integrate noisy sensory inputs towards a choice. We find that the observed complexity and functional roles of single neurons are readily understood in the framework of a dynamical process unfolding at the level of the population. The population dynamics can be reproduced by a trained recurrent neural network, which suggests a previously unknown mechanism for selection and integration of taskrelevant inputs. This mechanism indicates that selection and integration are two aspects of a single dynamical process unfolding within the same prefrontal circuits, and potentially provides a novel, general framework for understanding contextdependent computations.
Access options
Subscribe to Journal
Get full journal access for 1 year
$199.00
only $3.90 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
from$8.99
All prices are NET prices.
References
 1
Fuster, J. M. The Prefrontal Cortex 4th edn (Academic, 2008)
 2
Miller, E. K. & Cohen, J. D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001)
 3
Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995)
 4
Schroeder, C. E. & Lakatos, P. Lowfrequency neuronal oscillations as instruments of sensory selection. Trends Neurosci. 32, 9–18 (2009)
 5
Noudoost, B., Chang, M. H., Steinmetz, N. A. & Moore, T. Topdown control of visual attention. Curr. Opin. Neurobiol. 20, 183–190 (2010)
 6
Reynolds, J. H. & Chelazzi, L. Attentional modulation of visual processing. Annu. Rev. Neurosci. 27, 611–647 (2004)
 7
Maunsell, J. H. & Treue, S. Featurebased attention in visual cortex. Trends Neurosci. 29, 317–322 (2006)
 8
Fries, P. Neuronal gammaband synchronization as a fundamental process in cortical computation. Annu. Rev. Neurosci. 32, 209–224 (2009)
 9
Mansouri, F. A., Tanaka, K. & Buckley, M. J. Conflictinduced behavioural adjustment: a clue to the executive functions of the prefrontal cortex. Nature Rev. Neurosci. 10, 141–152 (2009)
 10
Tanji, J. & Hoshi, E. Role of the lateral prefrontal cortex in executive behavioral control. Physiol. Rev. 88, 37–57 (2008)
 11
Bruce, C. J. & Goldberg, M. E. Primate frontal eye fields. I. Single neurons discharging before saccades. J. Neurophysiol. 53, 603–635 (1985)
 12
Schall, J. D. The neural selection and control of saccades by the frontal eye field. Phil. Trans. R. Soc. Lond. B 357, 1073–1082 (2002)
 13
Moore, T. The neurobiology of visual attention: finding sources. Curr. Opin. Neurobiol. 16, 159–165 (2006)
 14
Kim, J. N. & Shadlen, M. N. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nature Neurosci. 2, 176–185 (1999)
 15
Machens, C. K., Romo, R. & Brody, C. D. Functional, but not anatomical, separation of “what” and “when” in prefrontal cortex. J. Neurosci. 30, 350–360 (2010)
 16
Rigotti, M. et al. The importance of mixed selectivity in complex cognitive tasks. Nature 497, 585–590 (2013)
 17
Stokes, M. G. et al. Dynamic coding for cognitive control in prefrontal cortex. Neuron 78, 364–375 (2013)
 18
Hernández, A. et al. Decoding a perceptual decision process across cortex. Neuron 66, 300–314 (2010)
 19
Churchland, M. M. et al. Neural population dynamics during reaching. Nature 487, 51–56 (2012)
 20
Shenoy, K. V., Sahani, M. & Churchland, M. M. Cortical control of arm movements: a dynamical systems perspective. Annu. Rev. Neurosci. 36, 337–359 (2013)
 21
Stopfer, M., Jayaraman, V. & Laurent, G. Intensity versus identity coding in an olfactory system. Neuron 39, 991–1004 (2003)
 22
Briggman, K. L., Abarbanel, H. D. & Kristan, W. B., Jr Optical imaging of neuronal populations during decisionmaking. Science 307, 896–901 (2005)
 23
Harvey, C. D., Coen, P. & Tank, D. W. Choicespecific sequences in parietal cortex during a virtualnavigation decision task. Nature 484, 62–68 (2012)
 24
Afshar, A. et al. Singletrial neural correlates of arm movement preparation. Neuron 71, 555–564 (2011)
 25
Sigala, N., Kusunoki, M., NimmoSmith, I., Gaffan, D. & Duncan, J. Hierarchical coding for sequential task events in the monkey prefrontal cortex. Proc. Natl Acad. Sci. USA 105, 11969–11974 (2008)
 26
Machens, C. K. Demixing population activity in higher cortical areas. Front. Comput. Neurosci. 4, 126 (2010)
 27
Shadlen, M. N. & Newsome, W. T. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 86, 1916–1936 (2001)
 28
Mazurek, M. E., Roitman, J. D., Ditterich, J. & Shadlen, M. N. A role for neural integrators in perceptual decision making. Cereb. Cortex 13, 1257–1269 (2003)
 29
Wang, X. J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002)
 30
Cohen, J. D., Dunbar, K. & McClelland, J. L. On the control of automatic processes: a parallel distributed processing account of the Stroop effect. Psychol. Rev. 97, 332–361 (1990)
 31
Deco, G. & Rolls, E. T. Attention and working memory: a dynamical model of neuronal activity in the prefrontal cortex. Eur. J. Neurosci. 18, 2374–2390 (2003)
 32
Sussillo, D. & Abbott, L. F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009)
 33
Sussillo, D. & Barak, O. Opening the black box: lowdimensional dynamics in highdimensional recurrent neural networks. Neural Comput. 25, 626–649 (2013)
 34
Zipser, D. & Andersen, R. A. A backpropagation programmed network that simulates response properties of a subset of posterior parietal neurons. Nature 331, 679–684 (1988)
 35
Martens, J. & Sutskever, I. Learning recurrent neural networks with hessianfree optimization. Proc. 28th Int. Conf. Machine Learn. (ICML, 2011)
 36
Churchland, A. K., Kiani, R. & Shadlen, M. N. Decisionmaking with multiple alternatives. Nature Neurosci. 11, 693–702 (2008)
 37
Reddi, B. A. & Carpenter, R. H. The influence of urgency on decision time. Nature Neurosci. 3, 827–830 (2000)
 38
Brunton, B. W., Botvinick, M. M. & Brody, C. D. Rats and humans can optimally accumulate evidence for decisionmaking. Science 340, 95–98 (2013)
 39
Seung, H. S. How the brain keeps the eyes still. Proc. Natl Acad. Sci. USA 93, 13339–13344 (1996)
 40
Goldman, M. S. Memory without feedback in a neural network. Neuron 61, 621–634 (2009)
 41
Sejnowski, T. J. On the stochastic dynamics of neuronal interaction. Biol. Cybern. 22, 203–211 (1976)
 42
Murphy, B. K. & Miller, K. D. Balanced amplification: a new mechanism of selective amplification of neural activity patterns. Neuron 61, 635–648 (2009)
 43
Ganguli, S., Huh, D. & Sompolinsky, H. Memory traces in dynamical systems. Proc. Natl Acad. Sci. USA 105, 18970–18975 (2008)
 44
Salinas, E. Contextdependent selection of visuomotor maps. BMC Neurosci. 5, 47 (2004)
 45
Zénon, A. & Krauzlis, R. J. Attention deficits without cortical neuronal deficits. Nature 489, 434–437 (2012)
 46
Roy, J. E., Riesenhuber, M., Poggio, T. & Miller, E. K. Prefrontal cortex activity during flexible categorization. J. Neurosci. 30, 8519–8528 (2010)
 47
Sasaki, R. & Uka, T. Dynamic readout of behaviorally relevant signals from area MT during task switching. Neuron 62, 147–157 (2009)
 48
Katzner, S., Busse, L. & Treue, S. Attention to the color of a moving stimulus modulates motionsignal processing in macaque area MT: evidence for a unified attentional system. Front. Syst. Neurosci. 3, 12 (2009)
 49
Machens, C. K., Romo, R. & Brody, C. D. Flexible control of mutual inhibition: a neural model of twointerval discrimination. Science 307, 1121–1124 (2005)
 50
Huk, A. C. & Meister, M. L. Neural correlates and neural computations in posterior parietal cortex during perceptual decisionmaking. Front. Integr. Neurosci. 6, 86 (2012)
Acknowledgements
We thank J. Powell, S. Fong and J. Brown for technical assistance, L. Abbott, for conversations on nonnormal dynamics, and L. Stryer, S. Hohl, S. Ganguli, M. Sahani, R. Kiani, C. Moore and T. Bhattacharya for discussions. V.M. and W.T.N. were supported by HHMI and the Air Force Research Laboratory (FA95500710537); D.S. and K.V.S. by an NIH Director’s Pioneer Award (1DP1OD006409) and DARPA REPAIR (N6600110C2010).
Author information
Affiliations
Contributions
V.M. and W.T.N. designed the study. V.M. collected the data. D.S. implemented the recurrent network. V.M. and D.S. analysed and modelled the data. V.M., D.S., K.V.S. and W.T.N. discussed the findings and wrote the paper.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Extended data figures and tables
Extended Data Figure 1 Recording locations and taskrelated patterns of population activity in PFC.
a, Recording locations (red dots) in monkey A are shown on anatomical magnetic resonance images in imaging planes that were oriented perpendicularly to the direction of electrode penetrations. Electrodes were lowered through a grid (1mm spacing) positioned over the arcuate sulcus (AS). Recordings covered the entire depth of the AS and extended rostrally onto the prearcuate gyrus and cortex near and lateral to the principal sulcus (PS). b–e, Representation of four task variables in the population response. Each multicoloured square corresponds to a recording location (red dots) in a. Within each square, each pixel corresponds to a unit recorded from that grid position, such that each square represents all the units recorded at the corresponding location. The colour of a pixel indicates the denoised regression coefficient of choice (b), motion coherence (c), colour coherence (d) and context (e) for a given unit (colour bars; grey: no units). These coefficients describe how much the trialbytrial firing rate of a given unit depends on the task variables in b–e. The position of each unit within a square is arbitrary; we therefore sorted them according to the amplitude of the coefficient of choice, which accounts for the diagonal bands of colour in b (topleft to bottomright, high to low choice coefficient). The positions of the pixels established in b are maintained in c–e, so that one can compare the amplitude of the coefficient for each task variable for every unit recorded from monkey A. Each of the four panels can be interpreted as the pattern of population activity elicited by the corresponding task variable. The four task variables elicit very distinct patterns of activity and are separable at the level of the population. Importantly, the coefficients were denoised with principal component analysis (see Supplementary Information, section 6.7) and can be estimated reliably from noisy neural responses (Extended Data Fig. 4i–l). Differences between activation patterns therefore reflect differences in the properties of the underlying units, not noise. f–j, Recording locations and taskrelated patterns of population activity for monkey F. Same conventions as in a–e. Recordings (f) covered the entire depth of the AS. The patterns of population activity elicited by a choice (g), by the motion evidence (h) and by context (j) are distinct, meaning that the representations of these task variables are separable at the level of the population. The representations of choice (g) and colour (i), however, are not separable in monkey F, indicating that colour inputs are processed differently in the two monkeys (see main text).
Extended Data Figure 2 Psychophysical performance for monkey F and for the model.
a–d, Psychophysical performance for monkey F, for motion (top) and colour contexts (bottom), averaged over 60 recording sessions (123,550 trials). Performance is shown as a function of motion (left) or colour (right) coherence in each behavioural context. As in Fig. 1c–f, coherence values along the horizontal axis correspond to the average low, intermediate and high motion coherence (a, c) and colour coherence (b, d) computed over all behavioural trials. The curves are fits of a behavioural model (see Supplementary Information, section 4). e–h, ‘Psychophysical’ performance for the trained neuralnetwork model (Figs 4–6) averaged over a total of 14,400 trials (200 repetitions per condition). Choices were generated based on the output of the model at the end of the stimulus presentation—an output larger than zero corresponds to a choice to the left target (choice 1), and an output smaller than zero corresponds to a choice to the left target (choice 2). We simulated model responses to inputs with motion and colour coherences of 0.03, 0.12 and 0.50. The variability in the input (that is, the variance of the underlying Gaussian distribution) was chosen such that the performance of the model for the relevant sensory signal qualitatively matches the performance of the monkeys. As in Fig. 1c–f, performance is shown as a function of motion (left) or colour (right) coherence in the motion (top) and colour contexts (bottom). Curves are fits of a behavioural model (as in a–d and in Fig. 1c–f). In each behavioural context, the relevant sensory input affects the model’s choices (e, h), but the irrelevant input does not (f, g), reflecting successful contextdependent integration. The model output essentially corresponds to the bounded temporal integral of the relevant input (not shown) and is completely unaffected by the irrelevant input.
Extended Data Figure 3 Mixed representation of task variables in PFC.
a–d, Example responses from six wellisolated single units in monkey A. Each column shows average normalized responses on correct trials for one of the single units. Responses are aligned to the onset of the randomdot stimulus, averaged with a 50ms sliding window, and sorted by one or more taskrelated variables (choice, motion coherence, colour coherence, context). The green lines mark time intervals with significant effects of choice (a), motion coherence (b), colour coherence (c), or context (d) as assessed by multivariable, linear regression (regression coefficient different from zero, P < 0.05). Linear regression and coefficient significance are computed over all trials (correct and incorrect, motion and colour context; Supplementary Information, section 6.3). The horizontal grey line corresponds to a normalized response equal to zero. a, Responses sorted by choice (solid, choice 1; dashed, choice 2) averaged over both contexts. b, Responses during motion context, sorted by choice and motion coherence (black to lightgrey, high to low motion coherence). c, Responses during colour context, sorted by choice and colour coherence (blue to cyan, high to low colour coherence). d, Responses sorted by choice and context (black, motion context; blue, colour context). As is typical for PFC, the activity of the example units depends on many task variables, indicating that they represent mixtures of the underlying task variables. e, f, Denoised regression coefficients for all units in monkey A (e) and monkey F (f). The data in Extended Data Fig. 1 are replotted here to directly compare the effects of different task variables (choice, motion, colour, context) to each other. Each data point corresponds to a unit, and the position along the horizontal and vertical axes is the denoised regression coefficient for the corresponding task variable. The horizontal and vertical lines in each panel intersect at the origin (0,0). Scale bars span the same range (0.1) in each panel. The different task variables are mixed at the level of individual units. Although units modulated by only one of the task variables do occur in the population, they do not form distinct clusters but rather are part of a continuum that typically includes all possible combinations of selectivities. Significant correlations between coefficients are shown in red (P < 0.05, Pearson’s correlation coefficient r).
Extended Data Figure 4 Targeted dimensionality reduction of population responses, and reliability of taskrelated axes and population trajectories.
a, Fraction of variance explained by the first 20 principal components of the responses in monkey A. Principal components are computed on correct trials only, on conditionaveraged responses. Conditions are defined on the basis of choice, motion coherence, colour coherence and context. Each time point of the average response for a given condition contributes an ‘independent’ sample for the principal components analysis, and variance is computed over conditions and times. b, Fraction of variance explained by the first 12 principal components. The total explainable variance (100%) is computed separately at each time, and reflects response differences across conditions. c, The four ‘taskrelated axes’ of choice, motion, colour and context expressed as linear combinations of the first 12 principal components. The four axes span a subspace containing the taskrelated variance in the population response (for example, Fig. 2 and Extended Data Fig. 6) and are obtained by orthogonalizing the denoised regression vectors for the corresponding task variables (see Supplementary Information, section 6.7; denoised regression coefficients are shown in Extended Data Figs 1 and 3e, f). The vertical axis in c corresponds to the projection of each axis onto a given principal component (that is, the contribution of that principal component to each axis). All four axes project onto multiple principal components and thus the corresponding task variables are mixed at the level of single principal components. d, Fraction of variance explained by the taskrelated axes of choice, motion, colour and context (solid lines), as in b. The four axes explain a larger fraction of the variance than the principal components at many times but, unlike the principal components, they do not explain the variance common to all conditions that is due to the passage of time (not shown). A possible concern with our analysis is that the time courses of variance explained in d could be misleading if the taskrelated axes, which we estimated only at a single time for each variable, are changing over time during the presentation of the random dots. Under this scenario, for example, the ‘humped’ shape of the motion input (solid black trace) might reflect a changing ensemble code for motion rather than actual changes in the strength of the motion signal in the neural population. To control for this possibility, we also computed timevarying ‘taskrelated axes’ by estimating the axes of motion, colour and context separately at each time throughout the 750ms dots presentation. The fractions of variance explained by the timevarying axes (dashed lines) and by the fixed axes (solid lines) have similar amplitudes and time courses. Thus, the effects of the corresponding task variables (during the presentation of the random dots) are adequately captured by the subspace spanned by the fixed axes (see Supplementary Information, section 6.8). e–h, Same as a–d, for monkey F. As shown in Extended Data Figs 1g, i and 3f (topright panel) the denoised regression coefficients of colour and choice are strongly correlated. As a consequence, the axis of colour explains only a small fraction of the variance in the population responses (h, blue; see main text). i–l, Reliability of taskrelated axes in monkey A. To determine to what extent variability (that is, noise) in single unit responses affects the taskrelated axes of choice, motion, colour and context (for example, Fig. 2 and Extended Data Fig. 6), we estimated each axis twice from two separate sets of trials (trial sets 1 and 2 in i–l). For each unit, we first assigned each trial to one of two subsets, and estimated denoised regression coefficients for the task variables separately for the two subsets. We then obtained taskrelated axes by orthogonalizing the corresponding denoised coefficients (see Supplementary Information, section 6.9). Here, the orthogonalized coefficients are computed both with (black) and without (grey) PCAbased denoising. The horizontal and vertical lines in each panel intersect at the origin (0,0). Scale bars span the same range (0.1) in each panel. Data points lying outside the specified horizontal or vertical plotting ranges are shown on the corresponding edges in each panel. i, Coefficients of choice. Each data point corresponds to the orthogonalized coefficient of choice for a given unit, computed from trials in set 1 (horizontal axis) or in set 2 (vertical axis). j–l, Same as i for the orthogonalized coefficients of motion (j), colour (k) and context (l). m–p, Orthogonalized regression coefficients for monkey F, as in i–l. Overall, after denoising the orthogonalized coefficients are highly consistent across the two sets of trials. Therefore, the observed differences in the activation pattern elicited by different task variables (Extended Data Fig. 1) are not due to the noisiness of neural responses, but rather reflect differences in the properties of the underlying units. q, r, Reliability of population trajectories. To assess the reliability of the trajectories in Fig. 2, we estimated the taskrelated axes and the resulting population trajectories (same conventions as Fig. 2) twice from two separate sets of trials (as i–l, see Supplementary Information, section 6.9). As in the example trajectories shown in q (trial set 1) and r (trial set 2), we consistently obtained very similar trajectories across the two sets of trials. To quantify the similarity between the trajectories from the two sets, we used trajectories obtained from one set to predict the trajectories obtained from the other set (see Supplementary Information, section 6.9). On average across 20 randomly defined pairs of trial sets, in both monkeys the population responses from one set explain 94% of the total variance in the responses of the other set (95% for the example in q and r). These numbers provide a lower bound on the true reliability of trajectories in Fig. 2, which are based on twice as many trials as those in q and r.
Extended Data Figure 5 Population responses along individual taskrelated axes.
a–e, Responses for monkey A. The average population responses on correct trials are replotted from Fig. 2, together with responses on a subset of incorrect trials (red curves). Here the responses are represented explicitly as a function of time (horizontal axis) and projected separately (vertical axes) onto the axes of choice (b), motion (c), colour (d) and context (e). As in Fig. 2, correct trials are sorted on the basis of context (motion: top subpanels; colour: bottom subpanels; see key in a), on the direction of the sensory evidence (filled, towards choice 1; dashed, towards choice 2) and strength of the sensory evidence (black to lightgrey, strongest to weakest motion; blue to cyan, strongest to weakest colour), and based on choice (thick, choice 1; thin, choice 2). Incorrect trials (red curves) are shown for the lowest motion coherence (during motion context, top left in b–e) and the lowest colour coherence (during colour context, bottom right in b–e). Vertical scale bars correspond to 1 unit of normalized response, and the horizontal lines are drawn at the same level in all four subpanels within b–e. a, Key to the condition averages shown in each panel of b–e, as well as to the corresponding statespace panels in Fig. 2. b, Projections of the population response onto the choice axis. Responses along the choice axis represent integration of evidence in both contexts. c, Projection onto the motion axis. Responses along the motion axis represent the momentary motion evidence during both motion (top left) and colour contexts (bottom left) (curves are parametrically ordered based on motion strength in both contexts), but not the colour evidence (right, curves are not ordered based on colour strength). d, Projection onto the colour axis. Responses along the colour axis represent the momentary colour evidence in the motion (top right) and colour contexts (bottom right) (ordered), but not the motion evidence (left, not ordered). e, Projection onto the context axis. Responses in the motion context (top, all curves above the horizontal line) and colour context (bottom, all curves below the horizontal line) are separated along the context axis, which maintains a representation of context. f–i, Responses for monkey F, same conventions as in b–e. The responses in f–i are also shown as trajectories in Extended Data Fig. 7g–l. The drift along the choice axis in Extended Data Fig. 7g–l is reflected in the overall positive slopes in f.
Extended Data Figure 6 Effect of context on PFC dynamics.
a, b, Responses from monkey A. Same conditions and conventions as in Fig. 2, but for activity projected into the twodimensional subspace capturing the variance due to choice (along the choice axis) and context (context axis). Components along the choice axis are enhanced relative to the context axis (see scale bars). The population response contains a representation of context, which is reflected in the separation between trajectories in the motion and colour contexts along the axis of context. The contextual signal is strongest early during the dots presentation. a, Effects of context (motion context versus colour context), choice (choice 1 versus choice 2), and motion input (direction and coherence, grey colours). b, Same trials as in a, but averaged to show the effect of the colour input (blue colours). c, d, Responses from monkey F, same conventions as in a, b. As in Extended Data Fig. 7a–f, we subtracted the acrosscondition average trajectory from each individual, raw trajectory (see Supplementary Information, section 6.10). The underlying raw population responses are shown in Extended Data Fig. 5f–i, and confirm that the representation of context is stable throughout the dots presentation time (Extended Data Fig. 5i).
Extended Data Figure 7 Dynamics of population responses in monkey F.
a–f, Response trajectories in the subspace spanned by the taskrelated axes of choice, motion and colour. Same conventions as in Fig. 2. Unlike in Fig. 2, here we subtracted the acrosscondition average trajectory from each individual, raw trajectory (see Supplementary Information, section 6.10). The raw trajectories are shown in g–l and the corresponding projections onto individual axes in Extended Data Fig. 5f–i. Three key features of the population responses are shared in monkey A (Fig. 2) and monkey F. First, movement along a single choice axis (a and f, red arrows) corresponds to integration of the relevant evidence in both contexts. Second, in both contexts the momentary motion evidence elicits responses along the axis of motion, which is substantially different from the axis of choice (a and d). Third, the motion evidence is strongly represented whether it is relevant (a) or irrelevant (d). Thus, the processing of motion inputs in both monkeys is inconsistent with current models of selection and integration (Fig. 3b–d). Unlike in monkey A, responses along the colour axis in monkey F (f and c) reflect the momentary colour evidence only weakly. The effects of colour on the trajectories in monkey F resemble the responses expected by the early selection model (Fig. 3b). g–l, Raw population responses. Population trajectories were computed and are represented as in Fig. 2. The trajectories in a–f were obtained by subtracting the acrosscondition average from each individual trajectory shown above. Overall, the responses have a tendency to move towards the left along the choice axis. An analogous, although weaker, overall drift can also be observed in monkey A, and contributes to the asymmetry between trajectories on choice 1 and choice 2 trials (Fig. 2). Because choice 1 corresponds to the target in the response field of the recorded neurons (see Supplementary Information, section 6.2), the drift reflects a tendency of individual firing rates to increase throughout the stimulus presentation time. By the definition of choice 1 and choice 2, a similar but opposite drift has to occur in neurons whose response field overlaps with choice 2 (the responses of which we did not record). In the framework of diffusiontobound models, such a drift can be interpreted as an urgency signal, which guarantees that the decision boundary is reached before the offset of the dots (refs 36, 37).
Extended Data Figure 8 Simulations of models of selective integration inconsistent with PFC responses.
We simulated population responses mimicking the observed PFC responses (a–c) and alternative responses expected based on the three models of contextdependent selection described in Fig. 3b–d (d–l) (see Supplementary Information, section 8). These simulations are based on a diffusiontobound model, unlike the simulations of the recurrent neural network models in Figs 5 and 6 and in Extended Data Figs 9 and 10e–s. Here, single neurons represent mixtures of three timedependent task variables of a diffusiontobound model, namely the momentary motion and colour evidence and the integrated relevant evidence. At the level of the population, these three task variables are represented along specific directions in state space (arrows in a, d, g, j; red, integrated evidence; black, momentary motion evidence; blue, momentary colour evidence). The four simulations differ only with respect to the direction and context dependence of the three task variables. We computed state space trajectories from the population responses using the targeted dimensionality reduction techniques discussed in the main text and in Supplementary Information. The resulting simulated population responses reproduce the schematic population responses in Fig. 3. a–c, Simulated population responses mimicking the observed PFC responses (Fig. 2). a, Response trajectories in the twodimensional subspace capturing the effects of choice and motion (left) or choice and colour (right) in the motion (top) and colour (bottom) contexts. Same conditions and conventions as in Fig. 2a, c and Fig. 2d, f. The three task variables are represented along three orthogonal directions in state space (arrows). b, Regression coefficients of choice, motion and colour for all simulated units in the population. For each unit, coefficients were computed with linear regression on all simulated trials (top) or separately on trials from the motion or colour context (bottom, context in parentheses). Scale bars represent arbitrary units. Numbers in the inset along each axis represent averages of the absolute value of the corresponding coefficients (±s.e.m., in parentheses). Significant correlations between coefficients are shown in red (P < 0.05, Pearson’s correlation coefficient r. c, Estimated strengths of the motion (top) and colour (bottom) inputs during motion (black) and colour (blue) contexts. Input strength is defined as the average of the absolute value of the corresponding regression coefficients. d–f, same as a–c, for simulated population responses expected from contextdependent early selection (Fig. 3b). When relevant, momentary motion (top) and colour (bottom) evidence are represented along the same direction as integrated evidence (arrows in d). g–i, same as a–c, for simulated population responses expected from contextdependent input directions (Fig. 3c). Integrated evidence is represented along the same direction in both contexts (red arrows in g). The relevant momentary evidence (motion in the motion context, top; colour in the colour context, bottom) is aligned with the direction of integration, whereas the irrelevant momentary evidence is orthogonal to it (black and blue arrows in g). j–l, same as a–c, for simulated population responses expected from contextdependent output directions (Fig. 3d). The momentary motion and colour evidence are represented along the same directions in both contexts (black and blue arrows in j). The direction of integration (red arrows in j) is aligned with the motion evidence in the motion context (top), and with the colour evidence in the colour context (bottom).
Extended Data Figure 9 Model population responses and validation of targeted dimensionality reduction.
a–e, Model population responses along individual taskrelated axes, same conventions as in Extended Data Fig. 5. Here we defined the taskrelated axes directly based on the synaptic connectivity in the model (see Supplementary Information, section 7.6; and panels h–j), rather than using the approximate estimates based on the population response (as for the PFC data, for example, Fig. 2). The same axes and the resulting projections underlie the trajectories in Fig. 5. The model integrates the contextually relevant evidence almost perfectly, and the responses along the choice axis (b) closely match the output of an appropriately tuned diffusiontobound model (not shown). Notably, nearperfect integration is not a core feature of the proposed mechanism of contextdependent selection (see main text, and Extended Data Fig. 10). f, g, Effect of context on model dynamics, same conditions and conventions as in Extended Data Fig. 6. Network activity is projected onto the twodimensional subspace capturing the variance due to choice (along the choice axis) and context (context axis). Same units on both axes (see scale bars). As in Fig. 5, fixed points of the dynamics (red crosses) and the associated right zeroeigenvectors (that is, the local direction of the line attractor, red lines) were computed separately for motion (top) and colour contexts (bottom) in the absence of sensory inputs. The line attractors computed in the two contexts, and the corresponding population trajectories, are separated along the context axis. f, Effects of context (motion context, colour context), choice (choice 1, choice 2) and motion input (direction and coherence, grey colours) on the population trajectories. g, Same trials as in f, but resorted and averaged to show the effect of the colour input (blue colours). The context axis is approximately orthogonal to the motion and colour inputs, and thus the effects of motion and colour on the population response (Fig. 5) are not revealed in the subspace spanned by the choice and context axes (f and g). h–j, Validation of targeted dimensionality reduction. To validate the dimensionality reduction approach used to analyse population responses in PFC (see Supplementary Information, sections 6.5–6.7), we estimated the regression vectors of choice, motion, colour and context from the simulated population responses (Fig. 5 and panels b–g) and compared them to the exactly known model dimensions that underlie the model dynamics (see definitions below). We estimated the regression vectors in three ways: by pooling responses from all model units and all trials (as in the PFC data, for example, Fig. 2 and Extended Data Fig. 6), or separately from the motion and colourrelevant trials (contexts). Orthogonalization of the regression vectors yields the taskrelated axes of the subspace of interest (for example, axes in Fig. 2). Most model dimensions (motion, colour and context inputs, and output) were defined by the corresponding synaptic weights after training. The line attractor, on the other hand, is the average direction of the right zeroeigenvector of the linearized dynamics around a fixed point, and was computed separately for the motion and colour contexts. h, The three regression vectors of motion (black arrows), plotted in the subspace spanned by the choice axis (that is, the regression vector of choice) and the motion axis (that is, the component of the regression vector of motion orthogonal to the choice axis). In the colour context, the motion regression vector closely approximates the actual motion input (black circle—the model dimension defined by synaptic weights). During the motion context, however, the motion regression vector has a strong component along the choice axis, reflecting the integration of motion evidence along that axis. The motion regression vector estimated from all trials corresponds to the average of the vectors from the two contexts; thus all three motion regression vectors lie in the same plane. i, The three regression vectors of colour (blue arrows) plotted in the subspace spanned by the choice and colour axes, analogous to h. The colour regression vector closely approximates the actual colour input (blue circle) in the motion context, but has a strong component along the choice axis in the colour context. Components along the motion (h) and colour (i) axes are scaled by a factor of 2 relative to those along the choice axis. j, Dot products (colour bar) between the regression vectors (horizontal axis) and the actual model dimensions (vertical axis), computed after setting all norms to 1. The choice regression vector closely approximates the direction of the line attractor in both contexts (squares labelled ‘1’). As shown also in h and i, the input regression vectors approximate the model inputs (defined by their synaptic weights) when the corresponding inputs are irrelevant (squares 2 and 4, motion and colour), whereas they approximate the line attractor when relevant (squares 3 and 5). Thus, the motion input is mostly contained in the plane spanned by the choice and motion axes (h), and the colour input is mostly contained in the plane spanned by the choice and colour axes (i). Finally, the single context regression vector is aligned with both context inputs (squares labelled 6), and closely approximates the difference between the two (not shown).
Extended Data Figure 10 Urgency and instability in the integration process.
a–d, Choice predictive neural activity (top) and psychometric curves (bottom) predicted by several variants of the standard diffusiontobound model (see Supplementary Information, section 7.7). a, Standard diffusiontobound model. Noisy momentary evidence is integrated over time until one of two bounds (+1 or −1; choice 1 or choice 2) is reached. The momentary evidence at each time point is drawn from a Gaussian distribution whose mean corresponds to the coherence of the input, and whose fixed variance is adjusted in each model to achieve the same overall performance (that is, similar psychometric curves, bottom panels). Coherences are 6%, 18% and 50% (the average colour coherences in monkey A, Fig. 1b). Average integrated evidence (neural firing rates, arbitrary units) is shown on choice 1 and choice 2 trials (thick versus thin) for evidence pointing towards choice 1 or choice 2 (solid versus dashed), on correct trials for all coherences (light grey to black, low to high coherence), and incorrect trials for the lowest coherence (red). The integrated evidence is analogous to the projection of the population response onto the choice axis (for example, Extended Data Fig. 5b, top left and bottom right). b, Urgency model. Here the choice is determined by a race between two diffusion processes (typically corresponding to two hemispheres), one with bound at +1, the other with bound at −1. The diffusion in each process is subject to a constant drift towards the corresponding bound, in addition to the drift provided by the momentary evidence. The inputindependent drift implements an ‘urgency’ signal, which guarantees that one of the bounds is reached within a short time. Only the integrated evidence from one of the diffusion processes is shown. The three ‘choice 1’ curves are compressed (in contrast to a) because the urgency signal causes the bound to be reached, and integration towards choice 1 to cease, more quickly than in a. In contrast, the ‘choice 2’ curves are not compressed as the diffusion process that accumulates evidence towards choice 1 never approaches a bound on these trials. c, Same as a, but here the diffusion process is subject to a drift away from the starting point (0) towards the closest bound (+1 or −1). The strength of the drift is proportional to the distance from the starting point, and creates an ‘instability’ at the starting point. d, Same as b, with an instability in the integration as in c for both diffusion processes. The asymmetry between choice 1 and choice 2 curves in b and d resembles the asymmetry in the corresponding PFC curves (Extended Data Figs 5b, f, upper left). e–j, Neural network model with urgency. This model is based on a similar architecture as the model in Fig. 4. Unlike the neural network in Fig. 4, which was trained solely based on the model output on the last time bin of the trial, here the network is trained based on the output it produces throughout the entire input presentation. The network was trained to reproduce the integrated evidence (that is, the decision variable) for one of the two diffusion processes (that is, one of the two ‘hemispheres’) in a diffusiontobound model with urgency (b, see Supplementary Information, section 7.7). Similar conventions as in Fig. 5. The urgency signal is controlled by an additional binary input into the network. Here, the urgency and sensory inputs are turned off as soon as a bound is reached. The network generates only a single, stable fixed point in each context, corresponding to the decision boundary (large red cross). The model also implements a series of points of relatively slow dynamics (small red crosses) approximately lying on a single curve. The axes of slow dynamics at these slow points (red lines) are locally aligned. Notably, responses at these slow points have a strong tendency to drift towards the single, stable fixed point (the decision boundary), and thus the curve of slow points does not correspond to an approximate line attractor. This drift implements the urgency signal and causes an asymmetry in the trajectories, which converge on a single point for choice 1, but have endpoints that are parametrically ordered by coherence along the choice axis for choice 2. As discussed below (panel r), this model relies on the same mechanism of selection as the original model (Fig. 5, see main text). k–p, Neural network model with instability. Trajectories show simulated population responses for a model (same architecture as in Fig. 4) that was trained to solve the contextdependent task (Fig. 1) only on highcoherence stimuli and in the absence of internal noise (see Supplementary Information, section 7.7). Same conventions as in Fig. 5. In the absence of noise, prolonged integration of evidence is not necessary for accurate performance on the task. As a consequence, the model implements a saddle point (blue cross) instead of an approximate line attractor. Points of slow dynamics (small red crosses, obscured by the red lines) occur only close to the saddle point. The right zeroeigenvectors of the linearized dynamics around these slow points (red lines) correspond to the directions of slowest dynamics, and determine the direction of the axis of choice. When displaced from the saddle point, the responses quickly drift towards one of the two stable attractors (large red crosses) corresponding to the choices. For a given choice, trajectories for all coherences therefore end in the same location along the choice axis, in contrast to the responses in the original model (Fig. 5). Despite these differences, the original model (Fig. 5) and the network model with instability (k–p) rely on a common mechanism of contextdependent selection (see panel s). q–s, Dynamical features (key, bottom) underlying input selection and choice in three related neural network models. All models are based on a common architecture (Fig. 4) but are the result of different training procedures. q, Dynamical features of the model described in the main paper (Figs 5 and 6), replotted from Fig. 6c. r, The urgency model (e–j). s, The instability model (k–p). In all models, the developing choice is implemented as more or less gradual movement along an axis of slow dynamics (specified by the locally computed right eigenvectors associated with the nearzero eigenvalue of the linearized dynamics, red lines). The inputs are selected, that is, result in movement along the axis of slow dynamics, depending on their projection onto the selection vector (the locally computed left eigenvectors associated with the nearzero eigenvalue). In this sense, the three models implement the same mechanisms of contextdependent selection and choice.
Supplementary information
Supplementary Information
This file contains Supplementary Text and Data sections 110 and Supplementary References. (PDF 2518 kb)
Rights and permissions
About this article
Cite this article
Mante, V., Sussillo, D., Shenoy, K. et al. Contextdependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013). https://doi.org/10.1038/nature12742
Received:
Accepted:
Published:
Issue Date:
Further reading

Awareness as inference in a higherorder state space
Neuroscience of Consciousness (2020)

Computational and neurophysiological principles underlying auditory perceptual decisions
Current Opinion in Physiology (2020)

Stimulusinduced sequential activity in supervisely trained recurrent networks of firing rate neurons
Nonlinear Dynamics (2020)

Dorsomedial prefrontal cortex and hippocampus represent strategic context even while simultaneously changing representation throughout a task session
Neurobiology of Learning and Memory (2020)

Prefrontal oscillations modulate the propagation of neuronal activity required for working memory
Neurobiology of Learning and Memory (2020)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.