Anterior cingulate cortex (ACC) is thought to control a wide range of reward, punishment, and uncertainty-related behaviors. However, how it does so is unclear. Here, in a Pavlovian procedure in which monkeys displayed a diverse repertoire of reward-related, punishment-related, and uncertainty-related behaviors, we show that many ACC-neurons represent expected value and uncertainty in a valence-specific manner, signaling value or uncertainty predictions about either rewards or punishments. Other ACC-neurons signal prediction information about rewards and punishments by displaying excitation to both (rather than excitation to one and inhibition to the other). This diversity in valence representations may support the role of ACC in many behavioral states that are either enhanced by reward and punishment (e.g., vigilance) or specific to either reward or punishment (e.g., approach and avoidance). Also, this first demonstration of punishment-uncertainty signals in the brain suggests that ACC could be a target for the treatment of uncertainty-related disorders of mood.
Actions to maximize rewards and minimize threats or punishments are thought to be controlled by a region of the prefrontal cortex called the anterior cingulate cortex (ACC), but how this control is accomplished is unclear. Currently, there are several views of how ACC controls motivated behavior1,2,3,4,5,6,7,8,9. Some posit that the ACC neurons rank-order outcomes (such as rewards and punishments) by their utility or value. The resulting signed value signal from ACC is then read out by other brain areas to guide action towards the best possible option or outcome2, 5,6,7,8, 10. This theory posits that many single ACC neurons carry information about both rewards and punishments, and rank-order predictions of these outcomes according to their expected value. Another theory posits that ACC controls negative affect, threat, and pain-driven behaviors. This theory suggests that many ACC neurons ought to preferentially signal punishment-related information11,12,13,14. A third theory suggests that the ACC is primarily engaged in deploying attention to monitor task-performance and detect errors in our predictions and actions15,16,17,18,19,20,21,22. This theory predicts that many ACC neurons ought to be activated by salient, engaging, and cognitively demanding situations, irrespective of their valence. Finally, a fourth theory based on imaging studies in humans identifies ACC as a central hub for processing information about uncertainty and other exploration-related and learning-related variables, such as the value of foraging23,24,25,26,27. Several recent proposals suggest that ACC could support some or even many of these functions1, 3, 4, 6, 7, 22, 27,28,29,30,31,32,33,34,35,36, however how it could do so is unclear.
One explanation that could reconcile these different theories is that the ACC may contain multiple circuits that could separately process control-related variables, such as information about reward, punishment, and uncertainty. Such diversity within ACC would allow other brain areas to read out inputs from the ACC flexibly, either by combining or multiplexing information about reward, punishment, and uncertainty, or by processing this information separately in a valence-specific manner.
To assess how single ACC neurons signal reward, punishment, and uncertainty, single-neuron activity was recorded from ACC while monkeys experienced certain and uncertain predictions about rewards and punishments. These predictions were cued by well-learned single visual fractals that served as conditioned stimuli (CS). This approach isolates value and uncertainty signals in the activity of single ACC neurons and avoids choice-related or learning-related fluctuations in subjective value and uncertainty37,38,39,40,41,42.
The results suggest that the ACC contains groups of neurons that process appetitive, aversive, salient, and uncertain information, over short and long time scales. Particularly, many ACC neurons represented expected value and uncertainty in a valence-specific manner, signaling value or uncertainty of predictions about either rewards or punishments. Other neurons signaled information about rewards and punishments by displaying excitation to both, rather than excitation to one and inhibition to the other. These data caution against a unified view of ACC and suggest that the ACC contains multiple circuits that could contribute to the top-down control of a wide range of reward, punishment, and uncertainty-related internal states and actions in complementary but partly distinct ways.
Diverse behaviors in response to rewards and punishments
To test how single ACC neurons (n = 329) signal information about reward, punishment, and uncertainty, two monkeys were conditioned with an appetitive–aversive behavioral procedure that contained two separate contexts, or blocks. One block contained 12 trials in which three visual fractal objects (CS) predicted rewards (juice) with 100, 50, and 0% chance. The second block contained 12 trials in which three visual fractal CSs predicted punishments (air puffs) with 100, 50, and 0% chance (Fig. 1a, b). The monkeys did not have to fixate the CSs to complete the trial (Methods). The design was such that the 100% reward CS had the highest value in the reward block and the 0% punishment CS had the highest value in the aversive block. Hypothetical encoding strategies of reward and punishment predictions are shown in Fig. 1c.
Analyses across all recorded behavioral sessions showed that the monkeys understood the behavioral procedure. Their CS-related anticipatory mouth movements (e.g., licking of the juice spout) and anticipatory blinking were correlated to the probability of reward and punishment, respectively (Spearman’s rank correlations; p < 0.001; rho = 0.24 for licking; rho = 0.37 for blinking; Supplementary Fig. 1). While these anticipatory responses were related to the expected value of the CSs, other behaviors, such as gaze, were driven by the absolute expected value of the CSs (often called motivational intensity or salience43) and by outcome uncertainty. Particularly, the monkeys’ gaze was initially drawn to the CSs associated with the more probable outcome (Fig. 1d), irrespective of valence. Target acquisition times during all the 100 and 50% CS trials were faster when compared with the 0% trials, within either reward or punishment block (p < 0.001; rank sum tests; single trials from no-abort cue blocks from all behavioral sessions = 31,838 trials). Within the reward block, target acquisition times were faster during 100% reward CS trials than during 50% reward CS trials (p < 0.01; rank sum tests).
Later in the trial the monkeys’ gaze was most strongly attracted towards the 50% reward CS (Fig. 1e; rank sum test; p < 0.001; measured during the last 1 s of the trial). We could not reliably observe CS-related gaze behavior during the last second of the punishment-predicting CSs because the gaze signal was quenched by defensive blinking (Fig. 1f).
To study punishment, it is crucial to verify that the outcome or unconditioned stimulus is aversive. It could be that blinking behaviors depicted in Fig. 1f reflect conditioning, but not aversion. To address this issue, we utilized distinct reward and punishment blocks that contained abort cues during one-half of the trials (Methods). This paradigm is a type of active avoidance used to test the aversiveness of cues, outcomes, and contexts in humans and experimental animals44,45,46,47,48. If monkeys made a saccade to the abort cue, the trial was aborted (Fig. 1g). Because these reward and punishment blocks alternate, the optimal reward-driven strategy would be to rapidly abort every trial in which the 100 and 50% reward CSs were not presented. In contrast, the monkeys’ aborting behaviors were influenced by reward, punishment, and uncertainty. In the punishment block, monkeys actively aborted punishment-predicting CSs more often than the 0% punishment CS, confirming that the air puff was an aversive outcome (Fig. 1h). While initially 100% punishment CS was associated with faster target acquisition than 0% CS (suggesting that it was more motivationally salient because it strongly attracted gaze; 43), later in the trial (when abort cues were presented), monkeys aborted the 100% CS faster than 50% and 0% punishment CSs. The data show that the monkeys’ motivation to abort was positively related to the probability of punishment.
Also, monkeys aborted 50% reward CSs less than 100% reward CS trials (though, note that the proportion of aborted trials during either 100 or 50% reward CSs was extremely low). This decrease in abort error rate is consistent with the observations in Fig. 1e that the 50% reward CS captured attention (despite having a lower expected value than 100% reward CS), reducing the number of saccades to the location of the abort-cue.
In sum, the behavioral data suggest that monkeys utilized different representations (or encoding strategies) of rewards and punishments to influence their behaviors.
Next, we asked if the ACC contains distinct representations of rewards and punishments or if ACC neurons encode rewards and punishments with a common currency, such as a general value signal (Fig. 1c). The locations of ACC neuronal recordings are shown in Supplementary Fig. 2 and match the locations of neuronal recordings in previous studies of macaque ACC5, 9, 32, 49, 50.
Many ACC cells signal value of either reward or punishment
The neuronal recordings revealed both that many ACC neurons represent expected value in a valence-specific manner, displaying greatest sensitivity to the probability of either rewards or punishments; and also that many ACC neurons prefer uncertain CSs in a valence-specific manner, often displaying preference for either reward or punishment uncertainty.
To summarize the valence-encoding strategies of ACC neurons, correlation analyses were peformed for each recorded neuron (n = 329), which assessed the relationship of CS responses and outcome probability (Methods; Supplementary Fig. 3 and Supplementary Table 1). The correlation coefficients are shown in Fig. 2b-inset. Most neurons that displayed significant correlations with outcome probability (Spearman’s rank correlation; statistical threshold: p < 0.05) did so in a valence-specific manner, for either reward or punishment probability.
Two valence-specific example neurons are shown in Fig. 2a. The first neuron displayed greatest excitation to punishment-predicting CSs and did not differentiate among 100, 50, and 0% reward CSs. The second neuron displayed greatest excitation to reward-predicting CSs (strongest to 100% reward) and did not differentiate among 100, 50, and 0% punishment CSs. To assess if such specific valence coding strategies were common among ACC neurons, we visualized the differences in neuronal responses for predictions of good outcomes and bad outcomes in the reward and punishment blocks (Fig. 2b). These analyses confirmed that many value-coding ACC neurons display valence-specific responses (Fig. 2).
Notably, other studies showed that within reward loss or reward gain trials, distinct ACC neurons signaled predictions and deliveries of reward gains and losses7, 28, 51, 52. Our data replicate these studies. Among reward-value neurons, many were excited or inhibited by increasing probabilities of rewards (Fig. 2c), signaling positive or negative reward values, respectively. In contrast, among the punishment-value neurons, most were excited by increasing probabilities of punishments.
Also, these ACC neurons did not display-trial-by-trial correlations with conditioned responses; see Supplementary Table 2.
The second most common valence coding strategy seen among ACC neurons signaled the motivational intensity or unsigned value of the CSs (red in Fig. 2). Few neurons showed a generalized value signal that rank-ordered the reward or punishment predictions according to their expected value (100 > 50 > 0% rewards and 0 > 50 > 100% punishments; Fig. 2). The time courses of neuronal responses among these neuronal types are shown in Supplementary Figs. 4–5.
Following 50% CSs, many ACC neurons discriminated between outcome deliveries and omissions. Many of these prediction error responses also tended to be valence specific. For example, distinct ACC neurons signaled reward omissions, reward deliveries, punishment omissions, and punishment deliveries (Supplementary Fig. 3). Interestingly, the value of the CSs (during the CS epoch) and their associated outcomes were often signaled by different neurons (Supplementary Fig. 3). To summarize, the data thus far suggest that ACC neurons can signal the values of predictions and deliveries of rewards and punishments through the activity of distinct populations of neurons.
ACC neurons are known to discriminate among different behavioral tasks33 and contexts53, and integrate task-related information over long time scales31, 54. Hence, it was important to assess if ACC contains valence-specific neurons in a behavioral procedure in which reward and punishment CSs are presented in the same context or block of time. To this end, in an additional control procedure, Monkey B was conditioned with nine distinct visual fractal CSs that predicted reward with 100, 75, 50, 25, and 0% chance and punishment with 100, 75, 50, and 25% chance. The monkey displayed conditioned responses that suggested that it understood the meaning of the CSs. The monkey’s licking and blinking behaviors were correlated with the probability of reward and punishment, respectively (Supplementary Fig. 6A). Among 95 ACC neurons recorded in this procedure, 16 were selectively enhanced by increases in the probability of punishment, but were insensitive to changes in the probability of reward. An example of one such neuron and their average responses are shown in Fig. 3a, b. Oppositely, 9/95 neurons were enhanced by increases in the probability of reward, but were insensitive to changes in the probability of punishment. An example of one such reward-specific value-coding neuron and their average activity are shown in Fig. 3c, d. Therefore, in the blocked reward-punishment behavioral procedure (Fig. 1) and in the single-block control procedure, the ACC contained neurons that signaled reward and punishment values in a valence-specific manner. Consistent with this observation, population coding strategies (summarized by plotting correlation coefficients of correlations that assessed the relationship of neuronal activity and probabilities of either reward or punishment) were qualitatively similar across the blocked and the single-block behavioral procedures (Fig. 2b; Supplementary Fig. 6B).
In the two-block appetitive/aversive procedure, monkeys and ACC neurons were highly sensitive to the nearing of their preferred context over many trials (Fig. 4). To show this, the analyses were concentrated on the behavioral and neuronal responses to the trial start cue. Though trial start cues were presented 3.5 s before the trial’s outcome would be experienced by the monkeys, their anticipatory orienting behavior (the duration it took them to foveate the trial start cue) was significantly faster in the reward vs. punishment block (Fig. 4a). Also, during the punishment block the speed of target acquisition changed as a function of trial number, decreasing as the reward block neared (Fig. 4a). Similarly, reward and punishment-sensitive neurons signaled the nearing of their preferred blocks. Reward-sensitive neurons displayed gradual and systematic changes in the aversive block in anticipation of the reward block (Fig. 4b). And, punishment-sensitive neurons displayed systematic changes in the reward block in anticipation of the aversive block (Fig. 4b).
Therefore, ACC neurons can signal valence specifically or non-specifically (Fig. 2). And, valence-specific neurons can signal the nearing of their preferred contexts or events over long time scales (Fig. 4).
Representation of reward and punishment uncertainty in ACC
Next, we assess if and how single ACC neurons encode uncertainty. In contrast with the hypothesis that ACC encodes economic or general common currency values, some recent neuroimaging studies have highlighted the ACC as a central hub for processing outcome uncertainty and guiding behaviors aimed to reduce this uncertainty. However, single neuron evidence for uncertainty processing in ACC has been missing.
We found 88 uncertainty selective neurons in the ACC (Methods), some of which responded for punishment uncertainty. An example neuron is shown in Fig. 5a (left). This neuron did not respond selectively to reward-predicting CSs or reward outcomes. In the punishment block, the neuron was most strongly activated by 50% punishment CS until the trial outcome. Among uncertainty selective neurons, 23/88 neurons were selectively excited by punishment uncertainty but not reward uncertainty (Fig. 5b; Supplementary Fig. 7). Zero were selectively inhibited by punishment uncertainty (Fig. 5b, c). Among the punishment-uncertainty neurons, nine neurons displayed significant variability (Kruskal–Wallis test; Methods) among the reward block CSs (Fig. 5c) but there was no obvious trend for positive or negative reward value coding among them. These results show that some ACC neurons signal punishment uncertainty in a selective manner, and that a minority of them can also multiplex this uncertainty signal with information about rewards.
Consistent with the observation that ACC can signal task-related information in a valence-specific manner (Fig. 2), we also found neurons that preferred reward uncertainty but not punishment uncertainty (Fig. 5a, b). An example neuron is shown in Fig. 5a-right. This neuron responded to the 50% reward CS and maintained tonic firing until the time of the trial-outcome (reward or no reward). It was not modulated by punishment predicting CSs. 45/88 uncertainty selective neurons were excited by reward uncertainty, while 12/88 uncertainty selective neurons were inhibited. Some reward uncertainty excited neurons also carried information about punishment probability; Fig. 5d and Supplementary Fig. 7. Among these, almost all were enhanced by increased probability of punishment. Finally, a minority of uncertainty selective neurons (8/88) were selective for both reward and punishment uncertainty (Fig. 5b).
Like all other populations of uncertainty or risk-coding neurons, ACC uncertainty neurons discriminated between 100 and 0% CSs in their preferred block37, 55,56,57,58. Within the aversive block, 9/23 punishment uncertainty neurons responded more to 100 than 0% punishment CSs, and 3/23 responded more to 0 than 100%. 22/45 reward uncertainty enhanced neurons responded more to 100 than 0% reward CSs, and 0/45 responded more to 0 than 100% reward CSs.
Outcome deliveries following uncertainty elicit prediction errors43, 59, 60. However, the majority of ACC uncertainty neurons did not signal prediction errors (Supplementary Fig. 8). This observation provides further evidence for the notion that within ACC, there may be distinct groups of neurons tracking predictions and outcome-related or feedback-related prediction errors (Supplementary Fig. 3).
In sum, ACC can signal uncertainty about either rewards or punishments in a valence-specific manner or to multiplex information about uncertainty and value in multiple manners (Supplementary Fig. 9). Also, prediction errors following uncertain epochs were often signaled by distinct populations of neurons.
Uncertainty can arise due to variability in outcomes or due to the possibility of making an error. For example, in our behavioral procedure aborting a reward-associated CS is an error because it reduces the probability of reward on that trial to 0. Though we observed few such errors (Fig. 1), the anticipation of the abort cue resulted in increases of overt attention towards the reward-possible CSs and the activity of uncertainty neurons in ACC (Supplementary Fig. 10). Therefore, ACC uncertainty neurons are sensitive to uncertainty arising due to internal processing (that increases attention27) as well as due to uncertain stimulus-outcome associations.
Recent studies show that several subcortical brain regions in the septum and the striatum contain populations of neurons that signal the graded level of reward uncertainty56, 57. Because these brain regions receive inputs from the ACC61, 62, an important question is, do ACC reward uncertainty neurons also signal graded levels of uncertainty and reward size? To answer this question, ACC reward enhanced uncertainty neurons were recorded while monkeys participated in the reward probability/reward amount behavioral procedures used in our previous studies55,56,57, 63. The reward-probability block contained five objects associated with five probabilistic reward predictions (0, 25, 50, 75 and 100% of 0.25 ml of juice). The reward-amount block contained five objects associated with certain reward predictions of varying reward amounts (0.25, 0.1875, 0.125, 0.065 and 0 ml). The expected values of the five CSs in the probability block matched the expected value of the five CSs in the amount block. The block design was used to remove the confounds introduced by risk-seeking related changes in subjective value processing of the CSs37, 55,56,57.
Consistent with our previous reports using the same procedure, after conditioning, monkeys choices rank ordered the CSs in either block according to their expected values (Fig. 6a, see refs 55,56,57, 63) indicating they understood the meanings of the CSs. After training, 58 ACC reward uncertainty enhanced neurons were identified and studied in the single CS reward probability/reward amount behavioral procedure.
The online identification of reward uncertainty neurons was the same as in our previous studies56, 57, 63. The average response of ACC reward uncertainty enhanced neurons is shown in Fig. 6b, c. During the reward-probability block, the neurons responded most strongly to the most uncertain CS (50% CS), more weakly to 25 and 75% CSs. The same neurons displayed the weakest response to the certain CSs (0 and 100%). In the reward amount block, their responses encoded reward size in a roughly linear manner, displaying highest firing for the greatest CS associated with the greatest reward sizes (Fig. 6c; rho = 0.3; p < 0.001; Spearman’s rank correlation; n = 58). These data indicate that, on average, ACC reward-uncertainty neurons signal the level of reward uncertainty and can also signal reward amounts in a roughly linear manner (Fig. 6b).
It has been proposed that the ACC exerts top–down control on a wide-range of reward, punishment, and uncertainty-related internal states and action selection processes. But how single ACC neurons encode reward, punishment, and uncertainty has remained unclear. Here, in a Pavlovian procedure in which monkeys displayed a diverse repertoire of reward, punishment, and uncertainty-related behaviors, many ACC neurons represented expected-value and uncertainty in a valence-specific manner, signaling value or uncertainty of either rewards or punishments. Also, other ACC neurons signaled information about rewards and punishments by displaying excitation to both, rather than excitation to one and inhibition to the other.
These results are mostly incompatible with several theories of ACC. First, though the ACC did contain neurons that were selectively activated by punishment predictions, there was no indication that the majority of ACC neurons exclusively signaled aversive or noxious information. Second, the data are inconsistent with the notion that many single ACC neurons encode a general value or common currency signal that generalizes across appetitive and aversive contexts (such as a theoretical economic value signal). Very few ACC neurons displayed positive relationships with reward probability and negative relationships with punishment probability (or vice versa).
In contrast, the data are consistent with the hypothesis that the ACC contains multiple circuits that have the capacity to distinctly process control-related and motivation-related variables, such as information about reward, punishment, and uncertainty. This diversity in information coding strategies could allow other brain areas to read-out inputs from the ACC flexibly, either by combining or multiplexing information about reward, punishment, and uncertainty; or by processing such information separately, to mediate many internal states and actions1,2,3,4, 22, 31, 33, 34, 49, 53, 64, such as those that are enhanced by both rewards and punishments (e.g., vigilance), and those specific to either reward or punishment (e.g., approach and avoidance).
This study demonstrates a novel selective punishment uncertainty signal in the brain and will therefore provide new opportunities to study internal states, such as anxiety and depression, which are thought to be strongly driven by persistent uncertainty about bad outcomes25, 65,66,67,68.
Punishment uncertainty neurons were often found in more anterior and ventral regions of the bank of the ACC while reward uncertainty neurons were found in all areas, but were particularly common in the dorsal extent of the ACC. Since the dorsal and ventral regions of the dorsal ACC have differences in their projection patterns to the striatum, amygdala, cortex, and brain stem62, 69,70,71, one possibility is that reward and punishment uncertainty are processed by distinct circuits involving somewhat different (but overlapping) areas of ACC. This conjecture is further supported by a series of experiments in which we identified neurons in two anatomically distinct networks that signal information about reward uncertainty but not punishment uncertainty: the septal-basal forebrain network and the striatal-pallidal network55,56,57, 63. Both of these networks receive inputs from and send direct and indirect inputs to the ACC, and neither displayed consistent coding of punishment uncertainty.
Previous work has identified that the ACC may be important for monitoring decision uncertainty and decision outcomes72 and for information seeking4, 73. It will be important to assess how different groups of neurons identified in this study, and throughout the subcortical uncertainty-related network, contribute to those functions.
When rewards and punishments are bundled into a single stimulus (or option) that monkeys can choose to approach or avoid, neurons in the pregenual cingulate cortex (an area that is anatomically distinct from but related to regions studied in this article) were either correlated with approach behavior or avoidance74. Interestingly, the avoidance-related neurons were also more active when the monkeys’ decision reaction times were slow. The authors suggested that this effect may be related to the subjective conflict between the reward and punishment within a particular bundle74. Along similar lines, Ebitz and Platt found that in a saccade task, distractor-induced conflict enhanced the activity of many neurons located within the ACC22. Here, it was found that the possibility of an abort cue presentation during high-valued trials (that monkeys rarely aborted) enhanced the activity of ACC uncertainty-sensitive neurons. In these trials, the abort cue was an aversive distractor to which saccades ought not to be made. One interesting hypothesis is that task-conflict, the increased probability of action-performance errors and of reward prediction errors (following uncertainty) modulate the same groups of ACC neurons.
ACC neurons display selectivity for contexts and tasks33, 53. Compatible with those observations was the finding that valence sensitive neurons predicted and anticipated the nearing of their preferred context over long time-scales. Also, on average, the CS responses of many reward and punishment-sensitive neurons did not consistently differ across blocks in which abort cues may be presented vs. blocks in which the abort cues were never presented, suggesting that ACC neurons may have the capacity to adjust their responses relative to the values of the possible predictions within a given context or block of trials.
Anterior regions of area 14c within the macaque ventromedial prefrontal cortex (vmPFC) and the dorsal raphe nucleus serotonin neurons, like ACC, display a capacity for context and valence specificity38, 75, 76 by tonically and phasically signaling the valence of blocks or contexts. One possibility that now requires serious investigation is that serotonin could play a crucial regulatory role in valence and context-specific behavioral selection through raphe-to-prefrontal projections to the ACC and vmPFC.
This study has several limitations. Rewards and punishments often elicit diverse and distinct behavioral states77, 78. This makes the dissociation of a valence-specific value signal from an action-value signal difficult. Though ACC neurons, like neurons in the orbitofrontal cortex42, did not display trial-by-trial correlations with conditioned responses, they likely influence action by increasing or decreasing motivation in a valence, context, and action-specific manner. Also, because one of the aims of this study was to ask if ACC contained uncertainty selective neurons, uncertainty that was unrelated to the CS was minimized. For example, the behavioral procedures did not include choice trials in which uncertainty or risk can come about due to many factors25, 37, 79, 80. It will be important to understand the time course and dynamics of reward and punishment-related signals in ACC during choice behavior, particularly in economic decision making tasks in which rewards and punishments are combined into single choice options.
Identifying neuronal mechanisms that facilitate flexible control of action and learning toward rewards and away from punishments remains a central pursuit of systems neuroscience. Here, we demonstrate a remarkable diversity and valence specificity of ACC neurons that may in part provide a neuronal substrate for such capacity. The data suggest that the ACC has multiple distinct and cooperative functions in behavioral control related to the expectation and receipt of reward, punishment, and their uncertainty.
Three adult male rhesus monkeys (Macaca mulatta) were used for the neurophysiology experiments (1—Monkey B, 2—Monkey Z, and 3—Monkey R). All procedures conformed to the Guide for the Care and Use of Laboratory Animals and were approved by the Washington University Institutional Animal Care and Use Committee. A plastic head holder and plastic recording chamber were fixed to the skull under general anesthesia and sterile surgical conditions. The chambers were tilted laterally by 35° and aimed at the anterior cingulate and the anterior regions of the basal ganglia. After the monkeys recovered from surgery, they participated in the behavioral and neurophysiological experiments.
While the monkeys participated in the behavioral procedure we recorded single neurons in the anterior cingulate. The recording sites were determined with 1 mm-spacing grid system and with the aid of MR images (3T) obtained along the direction of the recording chamber. This MRI-based estimation of neuron recording locations was aided by custom-built software (PyElectrode). Single-unit recording was performed using glass-coated electrodes (Alpha Omega). The electrode was inserted into the brain through a stainless-steel guide tube and advanced by an oil-driven micromanipulator (MO-97A, Narishige). Signal acquisition (including amplification and filtering) was performed using Alpha Omega 44 kHz SNR system. Action potential waveforms were identified online by multiple time-amplitude windows with an additional template matching algorithm (Alpha-Omega). Neuronal recording was restricted to single neurons that were isolated online. Neuronal and behavioral analyses were conducted offline in Matlab (Mathworks, Natick, MA).
Eye position was obtained with an infrared video camera (Eyelink, SR Research). Behavioral events and visual stimuli were controlled by Matlab (Mathworks, Natick, MA) with Psychophysics Toolbox extensions. Juice, used as reward, was delivered with a solenoid delivery reward system (CRIST Instruments). Juice-related licking was measured and quantified using previously described methods63. Airpuff (~35 psi) was delivered through a narrow tube placed ~6–8 cm from the monkey’s face.
The reward-punishment behavioral procedure consisted of two alternating blocks of trials: a reward block and a punishment block. In the reward block, three visual fractal CS were followed by a liquid reward (0.4 ml of juice) with 100, 50, and 0% chance, respectively. In the punishment block, three visual fractal CSs were followed by an air puff with 100, 50, and 0% chance, respectively. One block consisted of 12 trials with fixed proportions of trial types (each of the three CSs appears four times during each block). The inter-trial-intervals (ITIs) ranged from ~2–6 s.
Each trial started with the presentation of a trial-start cue at the center. The trial-start cue disappeared after 1 s and one of the three CSs was presented pseudo randomly (the CS could appear in three locations: 10 degrees to the left or to the right of the trial-start cue or in the center). After 2.5 s, the CS disappeared, and the outcome (juice, aifpuff, or nothing) was delivered. The monkeys were not required to fixate.
Abort cues were presented during one half of the trials in distinct blocks (abort-blocks) in one of three locations 10 degrees away from the CS. Abort blocks contained distinct visual fractals as CSs. Typically, a recording session contained the following repeating block structure: reward (abort) block, punishment (abort) block, reward block, punishment block, reward block, punishment block.
An additional single block reward-punishment behavioral procedure was used to study the activity of ACC in monkey B. The trial structure was the same as in the two-block procedure. Here, nine visual fractals served as CSs that predicted reward with 100, 75, 50, 25, and 0% chance and punishment with 100, 75, 50, and 25% chance. No abort cues were presented in this procedure.
To study if reward uncertainty-sensitive ACC neurons signal the level of reward uncertainty and reward size, a five reward-probability and reward-amount procedure was used. This procedure consisted of two blocks, a reward-probability block and a reward-amount block. The trial structure was detailed in White and Monosov, 2016 (see ref. 57). Each trial started with the presentation of a green trial-start cue at the center. The monkeys had to maintain fixation on the trial-start cue for 1 s; then the trial start cue disappeared and one of the CSs was presented pseudo randomly. After 2.5 s, the CS disappeared, and juice (if scheduled for that trial) was delivered. The reward-probability block contained five visual fractal objects CSs associated with five probabilistic reward predictions (0, 25, 50, 75 and 100% of 0.25 ml of juice). The reward-amount block contained five objects associated with certain reward predictions of varying reward amounts (0.25, 0.1875, 0.125, 0.065 and 0 ml). Each block consisted of 20 trials with fixed proportions of trial types (each of the five CSs appears four times in each block). Monkeys R, B, and Z participated in this procedure.
In separate experimental sessions, the monkeys’ choice preference was tested for the CSs. This procedure has been detailed in our previous studies56, 63. Each trial started with the presentation of the trial-start cue at the center, and the monkeys had to fixate it for 0.5 s. Then two CSs appeared 10 degrees to the left and right. The monkeys had to make a saccade to one of the two CSs within 5 s and fixate it for at least 750 ms. Then the unchosen CS disappeared, and after 1 s the outcome (associated with the chosen CS) was delivered, and the chosen CS disappeared. If the monkey failed to fixate one of the CSs, the trial was aborted and all stimuli disappeared. The trials were presented pseudo randomly, so that a block of 180 trials contained all possible combinations of the 10 CSs four times. Monkeys R, B, and Z participated in this procedure.
Data processing and statistics
Spike-density functions were generated by convolving spike times with a Gaussian filter (σ = 100 ms). All statistical tests were two-tailed. For comparisons between two task conditions for each neuron, we used a rank-sum test, unless otherwise noted. For comparisons between two task conditions across the population average, we used a paired signed-rank test, unless otherwise noted. p < 0.05 with Bonferroni correction was used as a threshold for these tests unless otherwise noted.
CS responses were measured in a single time window that started 100 ms from the CS onset until the outcome. Outcome responses were measured in a single 500-ms window that started 50 ms following the outcome delivery. To normalize task-event-related responses, we subtracted baseline activity (the last 500 ms of the inter-trial interval) from the activity during the task-event related measurement epoch.
A neuron was defined as CS responsive if it displayed variance across CSs either in the reward or punishment block (Kruskal–Wallis test, p < 0.01). The statistical identification of uncertainty neurons was detailed previously38, 56, 57, 63. As before, a neuron was defined as uncertainty sensitive if its CS responses varied across the three possible outcome predictions in either block (Kruskal–Wallis test, p < 0.01) and if its response to the uncertain CS (50%) was significantly stronger or weaker than its responses to both 100 and 0% CSs (two-tailed rank-sum test; p < 0.05; Bonferroni corrected). Therefore, a neuron could be uncertainty-sensitive in reward block, punishment block, or both.
To calculate receiver operating characteristic (ROC) that assessed neuronal discrimination of value and uncertainty, we compared spike counts during the CS epoch of two conditions. The analysis was structured so that area under curve values >0.5 indicate that the response for the value or uncertainty was a selective enhancement, while values <0.5 indicate that the response for the value or uncertainty was a selective suppression.
Trial-by-trial correlations of single unit activity and conditioned responses (Supplementary Table 2) were performed so as to minimize the influence of global task correlations. For each neuron type, the neurons’ preferred CS was chosen (e.g., reward uncertainty enhanced neurons’ 50% reward CS epochs were analysed), and the relationship of spiking activity (mean spike count) and the magnitude of conditioned responses during that CS epoch was tested by a Spearman’s rank correlation. Similar results were obtained if partial correlations were performed across all CS conditions in which spiking activity and conditioned responses were z-scored within each CS condition before the correlation was performed. For the trial-by-trial correlations assessing firing rate during the CS epoch and the subsequent ITI pupil dilation, the pupils were assessed in a 0.50 s window, 1.5 s following the outcome. For each neuron, object selectivity was assessed by comparing its responses to two distinct visual fractals that conveyed the neuron’s preferred outcome prediction (e.g., reward value enhanced neurons’ responses for two visual fractals that conveyed 100% reward).
Supplementary Fig. 9 visualizes how uncertainty and value-coding strategies co-occur in single neurons across the reward and punishment blocks. Each neuron is represented by a vector of 4 value and uncertainty discrimination indices ranging from 0 to 1. Value discrimination indices were obtained by ROC analyses that compared 100 and 0% CS responses. Uncertainty discrimination indices were obtained by ROC analyses that compared 50 and 100% CS responses. If a neuron did not pass the uncertainty sensitivity test (described above), the result of the uncertainty ROC analysis was set to 0.5. If a neuron was uncertainty-sensitive in one of the blocks (i.e., reward or punishment blocks), its value index was set to 0.5 in that block. The data were organized by hierarchical clustering on Euclidean distance utilizing Ward’s minimum variance method. Because clustering is a data mining method, here it is utilized to organize the matrix of neurons for qualitative observation. The results are interpreted as visual confirmation for multiple value and uncertainty coding strategies in ACC, not as evidence for discrete and stable clusters (e.g., robust to noise or increases or decreases in neuron number).
Data supporting the findings of this study are available within the article and its Supplementary Information Figures or from the authors on request.
Heilbronner, S. R. & Hayden, B. Y. Dorsal anterior cingulate cortex: a bottom-up view. Annu. Rev. Neurosci. 39, 149–170 (2016).
Shenhav, A., Botvinick, M. M. & Cohen, J. D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).
Shenhav, A., Cohen, J. D. & Botvinick, M. M. Dorsal anterior cingulate cortex and the value of control. Nat. Neurosci. 19, 1286–1291 (2016).
Kolling, N. et al. Value, search, persistence and model updating in anterior cingulate cortex. Nat. Neurosci. 19, 1280–1285 (2016).
Cai, X. & Padoa-Schioppa, C. Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex. J. Neurosci. 32, 3791–3808 (2012).
Wallis, J. D. & Kennerley, S. W. Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198 (2010).
Kennerley, S. W., Behrens, T. E. & Wallis, J. D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581–1589 (2011).
Rudebeck, P. H. et al. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 28, 13775–13785 (2008).
Rudebeck, P. H., Bannerman, D. M. & Rushworth, M. F. The contribution of distinct subregions of the ventromedial frontal cortex to emotion, social behavior, and decision making. Cogn. Affect. Behav. Neurosci. 8, 485–497 (2008).
Kennerley, S. W., Walton, M. E., Behrens, T. E., Buckley, M. J. & Rushworth, M. F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006).
Rainville, P., Duncan, G. H., Price, D. D., Carrier, B. & Bushnell, M. C. Pain affect encoded in human anterior cingulate but not somatosensory cortex. Science 277, 968–971 (1997).
Wiech, K. & Tracey, I. The influence of negative emotions on pain: behavioral effects and neural mechanisms. Neuroimage 47, 987–994 (2009).
Iwata, K. et al. Anterior cingulate cortical neuronal activity during perception of noxious thermal stimuli in monkeys. J. Neurophysiol. 94, 1980–1991 (2005).
Kwan, C. L., Crawley, A. P., Mikulis, D. J. & Davis, K. D. An fMRI study of the anterior cingulate cortex and surrounding medial wall activations evoked by noxious cutaneous heat and cold stimuli. Pain 85, 359–374 (2000).
Hayden, B. Y., Heilbronner, S. R., Pearson, J. M. & Platt, M. L. Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J. Neurosci. 31, 4178–4187 (2011).
Ito, S., Stuphorn, V., Brown, J. W. & Schall, J. D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302, 120–122 (2003).
Carter, C. S. et al. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280, 747–749 (1998).
Botvinick, M., Nystrom, L. E., Fissell, K., Carter, C. S. & Cohen, J. D. Conflict monitoring versus selection-for-action in anterior cingulate cortex. Nature. 402, 179–181 (1999).
van Veen, V., Cohen, J. D., Botvinick, M. M., Stenger, V. A. & Carter, C. S. Anterior cingulate cortex, conflict monitoring, and levels of processing. Neuroimage. 14, 1302–1308 (2001).
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S. & Cohen, J. D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001).
Brown, J. W. & Braver, T. S. Learned predictions of error likelihood in the anterior cingulate cortex. Science 307, 1118–1121 (2005).
Ebitz, R. B. & Platt, M. L. Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal. Neuron. 85, 628–640 (2015).
Kolling, N., Behrens, T. E., Mars, R. B. & Rushworth, M. F. Neural mechanisms of foraging. Science 336, 95–98 (2012).
Behrens, T. E., Woolrich, M. W., Walton, M. E. & Rushworth, M. F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).
Hirsh, J. B., Mar, R. A. & Peterson, J. B. Psychological entropy: a framework for understanding uncertainty-related anxiety. Psychol. Rev. 119, 304–320 (2012).
Burke, C. J. & Tobler, P. N. Reward skewness coding in the insula independent of probability and loss. J. Neurophysiol. 106, 2415–2422 (2011).
Bryden, D. W., Johnson, E. E., Tobia, S. C., Kashtelyan, V. & Roesch, M. R. Attention for learning signals in anterior cingulate cortex. J. Neurosci. 31, 18266–18274 (2011).
Quilodran, R., Rothe, M. & Procyk, E. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57, 314–325 (2008).
Wallis, J. D. & Rich, E. L. Challenges of interpreting frontal neurons during value-based decision-making. Front. Neurosci. 5, 124 (2011).
Wittmann, M. K. et al. Predictive decision making driven by multiple time-linked reward representations in the anterior cingulate cortex. Nat. Commun. 7, 12327 (2016).
Kolling, N., Behrens, T., Wittmann, M. K. & Rushworth, M. Multiple signals in anterior cingulate cortex. Curr. Opin. Neurobiol. 37, 36–43 (2016).
Hayden, B. Y., Pearson, J. M. & Platt, M. L. Neuronal basis of sequential foraging decisions in a patchy environment. Nat. Neurosci. 14, 933–939 (2011).
Hosokawa, T., Kennerley, S. W., Sloan, J. & Wallis, J. D. Single-neuron mechanisms underlying cost-benefit analysis in frontal cortex. J. Neurosci. 33, 17385–17397 (2013).
Dosenbach, N. U. et al. A core system for the implementation of task sets. Neuron 50, 799–812 (2006).
Dosenbach, N. U. et al. Distinct brain networks for adaptive and stable task control in humans. Proc. Natl. Acad. Sci. USA 104, 11073–11078 (2007).
Neta, M., Nelson, S. M. & Petersen, S. E. Dorsal anterior cingulate, medial superior frontal cortex, and anterior insula show performance reporting-related late task control signals. Cereb. Cortex 10.1093/cercor/bhw053 (2016).
O’Neill, M. & Schultz, W. Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68, 789–800 (2010).
Monosov, I. E. & Hikosaka, O. Regionally distinct processing of rewards and punishments by the primate ventromedial prefrontal cortex. J. Neurosci. 32, 10318–10330 (2012).
Paton, J. J., Belova, M. A., Morrison, S. E. & Salzman, C. D. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439, 865–870 (2006).
Matsumoto, M. & Hikosaka, O. Representation of negative motivational value in the primate lateral habenula. Nat. Neurosci. 12, 77–84 (2009).
Kim, H. F. & Hikosaka, O. Distinct basal ganglia circuits controlling behaviors guided by flexible and stable values. Neuron 79, 1001–1010 (2013).
Morrison, S. E. & Salzman, C. D. Representations of appetitive and aversive information in the primate orbitofrontal cortex. Ann. N Y Acad. Sci. 1239, 59–70 (2011).
Matsumoto, M. & Hikosaka, O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837–841 (2009).
Mobbs, D. et al. When fear is near: threat imminence elicits prefrontal-periaqueductal gray shifts in humans. Science 317, 1079–1083 (2007).
Pang, K. C., Jiao, X., Sinha, S., Beck, K. D. & Servatius, R. J. Damage of GABAergic neurons in the medial septum impairs spatial working memory and extinction of active avoidance: effects on proactive interference. Hippocampus 21, 835–846 (2011).
Delgado, M. R., Jou, R. L., Ledoux, J. E. & Phelps, E. A. Avoiding negative outcomes: tracking the mechanisms of avoidance learning in humans during fear conditioning. Front. Behav. Neurosci. 3, 33 (2009).
Rosen, J. B. & Donley, M. P. Animal studies of amygdala function in fear and uncertainty: relevance to human research. Biol. Psychol. 73, 49–60 (2006).
LeDoux, J. Fear and the brain: where have we been, and where are we going? Biol. Psychiatry 44, 1229–1238 (1998).
Luk, C. H. & Wallis, J. D. Choice coding in frontal cortex during stimulus-guided or action-guided decision-making. J. Neurosci. 33, 1864–1871 (2013).
Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).
Sallet, J. et al. Expectations, gains, and losses in the anterior cingulate cortex. Cogn. Affect. Behav. Neurosci. 7, 327–336 (2007).
Kennerley, S. W. & Wallis, J. D. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur. J. Neurosci. 29, 2061–2073 (2009).
Saez, A. et al. Abstract context representations in primate amygdala and prefrontal cortex. Neuron 87, 869–881 (2015).
Kawai, T. et al. Roles of the lateral habenula and anterior cingulate cortex in negative outcome monitoring and behavioral adjustment in nonhuman primates. Neuron 88, 792–804 (2015).
Ledbetter, M. N., Chen, D. C. & Monosov, I. E. Multiple mechanisms for processing reward uncertainty in the primate basal forebrain. J. Neurosci. 10.1523/JNEUROSCI.1123-16.2016 (2016).
Monosov, I. E. & Hikosaka, O. Selective and graded coding of reward uncertainty by neurons in the primate anterodorsal septal region. Nat. Neurosci. 16, 756–762 (2013).
White, J. K. & Monosov, I. E. Neurons in the primate dorsal striatum signal the uncertainty of object–reward associations. Nat. Commun. 7, 12735 (2016).
Burke, C. J. & Tobler, P. N. Coding of reward probability and risk by single neurons in animals. Front. Neurosci. 5, 121 (2012).
Schultz, W. Dopamine signals for reward value and risk: basic and recent data. Behav. Brain Funct. 6, 124 (2010).
Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Sheehan, T. P., Chambers, R. A. & Russell, D. S. Regulation of affect by the lateral septum: implications for neuropsychiatry. Brain Res. Brain Res. Rev. 46, 71–117 (2004).
Haber, S. N., Kunishio, K., Mizobuchi, M. & Lynd-Balta, E. The orbital and medial prefrontal circuit through the primate basal ganglia. J. Neurosci. 15, 4851–4867 (1995).
Monosov, I. E., Leopold, D. A. & Hikosaka, O. Neurons in the primate medial basal forebrain signal combined information about reward uncertainty, value, and punishment anticipation. J. Neurosci. 35, 7443–7459 (2015).
Rudebeck, P. H., Buckley, M. J., Walton, M. E. & Rushworth, M. F. A role for the macaque anterior cingulate gyrus in social valuation. Science 313, 1310–1312 (2006).
Steimer, T. The biology of fear-and anxiety-related behaviors. Dialogues Clin. Neurosci. 4, 231–250 (2002).
Craig, K., Brown, K. & Baum, A. Environmental factors in the etiology of anxiety. Psychopharmacology 1325–1339 (Raven Press, 1995).
Imada, H. & Nageishi, Y. The concept of uncertainty in animal experiments using aversive stimulation. Psychol. Bull. 91, 573 (1982).
Grupe, D. W. & Nitschke, J. B. Uncertainty and anticipation in anxiety: an integrated neurobiological and psychological perspective. Nat. Rev. Neurosci. 14, 488–501 (2013).
Takada, M. et al. Organization of inputs from cingulate motor areas to basal ganglia in macaque monkey. Eur. J. Neurosci. 14, 1633–1650 (2001).
An, X., Bandler, R., Ongur, D. & Price, J. L. Prefrontal cortical projections to longitudinal columns in the midbrain periaqueductal gray in macaque monkeys. J. Comp. Neurol. 401, 455–479 (1998).
Vogt, B. A. & Pandya, D. N. Cingulate cortex of the rhesus monkey: II. Cortical afferents. J. Comp. Neurol. 262, 271–289 (1987).
Khamassi, M., Quilodran, R., Enel, P., Dominey, P. F. & Procyk, E. Behavioral regulation and the modulation of information coding in the lateral prefrontal and cingulate cortex. Cereb. Cortex 25, 3197–3218 (2015).
Stoll, F. M., Fontanier, V. & Procyk, E. Specific frontal neural dynamics contribute to decisions to check. Nat. Commun. 7, 11990 (2016).
Amemori, K. & Graybiel, A. M. Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat. Neurosci. 15, 776–785 (2012).
Cohen, J. Y., Amoroso, M. W. & Uchida, N. Serotonergic neurons signal reward and punishment on multiple timescales. eLife 10.7554/eLife.06346 (2015).
Hayashi, K., Nakao, K. & Nakamura, K. Appetitive and aversive information coding in the primate dorsal raphe nucleus. J. Neurosci. 35, 6195–6208 (2015).
Seymour, B., Singer, T. & Dolan, R. The neurobiology of punishment. Nat. Rev. Neurosci. 8, 300–311 (2007).
Namburi, P., Al-Hasani, R., Calhoon, G. G., Bruchas, M. R. & Tye, K. M. Architectural representation of valence in the limbic system. Neuropsychopharmacology 41, 1697–1715 (2016).
Bach, D. R. & Dolan, R. J. Knowing how much you don’t know: a neural organization of uncertainty estimates. Nat. Rev. Neurosci. 13, 572–586 (2012).
Izawa, J. & Shadmehr, R. On-line processing of uncertain information in visuomotor control. J. Neurosci. 28, 11360–11368 (2008).
This work was supported by the Defense Advanced Research Projects Agency (DARPA) Biological Technologies Office (BTO) ElectRx program under the auspices of Dr. Doug Weber through the CMO Grant/Contract No.HR0011-16-2-0022 and the National Institute of Mental Health under Award Number R01MH110594. I am grateful to Mr. J. Kael White, Ms. Kim Kocher, and Ms. Ying Jiang for assisting in data acquisition, to Ms. Kim Kocher for fantastic animal care and training, and to Ms. Katherine Conen, Mr. J. Kael White, Mr. Ben Acland, and Mr. Mike Traner for reading earlier versions of this manuscript. The content in this article is solely the responsibility of the author and does not necessarily represent the official views of its funders.
The author declares no competing financial interests.
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
About this article
Cite this article
Monosov, I.E. Anterior cingulate is a source of valence-specific information about value and uncertainty. Nat Commun 8, 134 (2017). https://doi.org/10.1038/s41467-017-00072-y
Interactions between ventrolateral prefrontal and anterior cingulate cortex during learning and behavioural change
Cortical preparatory activity during motor learning reflects visuomotor retention deficits after punishment feedback
Experimental Brain Research (2021)
Computational Brain & Behavior (2021)