The direction of the eye gaze of others is a prominent social cue in primates and is important for communication1,2,3,4,5,6,7,8,9,10,11. Although gaze can signal threat and elicit anxiety6,12,13, it remains unclear whether it shares neural circuitry with stimulus value. Notably, gaze not only has valence, but can also serve as a predictor of the outcome of a social encounter, which can be either negative or positive2,8,12,13. Here we show that the neural codes for gaze and valence overlap in primates and that they involve two different mechanisms: one for the outcome and another for its expectation. Monkeys participated in the human intruder test13,14, in which a human participant had either a direct or averted gaze, interleaved with blocks of aversive and appetitive conditioning. We find that single neurons in the amygdala encode gaze15, whereas neurons in the anterior cingulate cortex encode the social context16, but not gaze. We identify a shared population in the amygdala for which the neural responses to direct and averted gaze parallel the responses to aversive and appetitive stimulus, respectively. Furthermore, we distinguish between two neural mechanisms—an overall-activity scheme that is used for gaze and the unconditioned stimulus, and a correlated-selectivity scheme that is used for gaze and the conditioned stimulus. These findings provide insights into the origins of the neural mechanisms that underlie the computations of both social interactions and valence, and could help to shed light on mechanisms that underlie social anxiety and the comorbidity between anxiety and impaired social interactions.
Subscribe to Journal
Get full journal access for 1 year
only $3.90 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
All data supporting the findings of this study are available from the corresponding author upon reasonable request.
Custom code for behavioural and electrophysiological tests is available from the corresponding author upon reasonable request.
Jones, W. & Klin, A. Attention to eyes is present but in decline in 2–6-month-old infants later diagnosed with autism. Nature 504, 427–431 (2013).
Emery, N. J. The eyes have it: the neuroethology, function and evolution of social gaze. Neurosci. Biobehav. Rev. 24, 581–604 (2000).
Gobel, M. S., Kim, H. S. & Richardson, D. C. The dual function of social gaze. Cognition 136, 359–364 (2015).
Adolphs, R. Neural systems for recognizing emotion. Curr. Opin. Neurobiol. 12, 169–177 (2002).
Zhou, Y. et al. Atypical behaviour and connectivity in SHANK3-mutant macaques. Nature 570, 326–331 (2019).
Schneier, F. R., Kent, J. M., Star, A. & Hirsch, J. Neural circuitry of submissive behavior in social anxiety disorder: a preliminary study of response to direct eye gaze. Psychiatry Res. 173, 248–250 (2009).
Schneier, F. R., Rodebaugh, T. L., Blanco, C., Lewin, H. & Liebowitz, M. R. Fear and avoidance of eye contact in social anxiety disorder. Compr. Psychiatry 52, 81–87 (2011).
Ballesta, S. & Duhamel, J. R. Rudimentary empathy in macaques’ social decision-making. Proc. Natl Acad. Sci. USA 112, 15516–15521 (2015).
Gariépy, J. F. et al. Social learning in humans and other animals. Front. Neurosci. 8, 58 (2014).
Rutishauser, U. et al. Single-neuron correlates of atypical face processing in autism. Neuron 80, 887–899 (2013).
Hadjikhani, N. et al. Look me in the eyes: constraining gaze in the eye-region provokes abnormally high subcortical activation in autism. Sci. Rep. 7, 3163 (2017).
Shepherd, S. V. Following gaze: gaze-following behavior as a window into social cognition. Front. Integr. Neurosci. 4, 5 (2010).
Kalin, N. H. & Shelton, S. E. Defensive behaviors in infant rhesus monkeys: environmental cues and neurochemical regulation. Science 243, 1718–1721 (1989).
Oler, J. A. et al. Amygdalar and hippocampal substrates of anxious temperament differ in their heritability. Nature 466, 864–868 (2010).
Mosher, C. P., Zimmerman, P. E. & Gothard, K. M. Neurons in the monkey amygdala detect eye contact during naturalistic social interactions. Curr. Biol. 24, 2459–2464 (2014).
Haroush, K. & Williams, Z. M. Neuronal prediction of opponent’s behavior during cooperative social interchange in primates. Cell 160, 1233–1245 (2015).
Gamer, M. & Büchel, C. Amygdala activation predicts gaze toward fearful eyes. J. Neurosci. 29, 9123–9126 (2009).
Gothard, K. M., Battaglia, F. P., Erickson, C. A., Spitler, K. M. & Amaral, D. G. Neural responses to facial expression and face identity in the monkey amygdala. J. Neurophysiol. 97, 1671–1683 (2007).
Adolphs, R. What does the amygdala contribute to social cognition? Ann. NY Acad. Sci. 1191, 42–61 (2010).
Stein, M. B. & Stein, D. J. Social anxiety disorder. Lancet 371, 1115–1125 (2008).
Tovote, P., Fadok, J. P. & Lüthi, A. Neuronal circuits for fear and anxiety. Nat. Rev. Neurosci. 16, 317–331 (2015).
Herry, C. & Johansen, J. P. Encoding of fear learning and memory in distributed neuronal circuits. Nat. Neurosci. 17, 1644–1654 (2014).
Duvarci, S. & Pare, D. Amygdala microcircuits controlling learned fear. Neuron 82, 966–980 (2014).
Janak, P. H. & Tye, K. M. From circuits to behaviour in the amygdala. Nature 517, 284–292 (2015).
Putnam, P. T. & Gothard, K. M. Multidimensional neural selectivity in the primate amygdala. eNeuro 6, ENEURO.0153-19.2019 (2019).
Kyriazi, P., Headley, D.B. & Pare, D. Multi-dimensional coding by basolateral amygdala neurons. Neuron 99, 1315–1328 (2018).
Pryluk, R., Kfir, Y., Gelbard-Sagiv, H., Fried, I. & Paz, R. A tradeoff in the neural code across regions and species. Cell 176, 597–6098 (2019).
Munuera, J., Rigotti, M. & Salzman, C. D. Shared neural coding for social hierarchy and reward value in primate amygdala. Nat. Neurosci. 21, 415–423 (2018).
Dunbar, R. I. M. The social brain hypothesis. Evol. Anthropol. 6, 178–190 (1998).
Myllyneva, A., Ranta, K. & Hietanen, J. K. Psychophysiological responses to eye contact in adolescents with social anxiety disorder. Biol. Psychol. 109, 151–158 (2015).
Bickart, K. C., Wright, C. I., Dautoff, R. J., Dickerson, B. C. & Barrett, L. F. Amygdala volume and social network size in humans. Nat. Neurosci. 14, 163–164 (2011).
Sallet, J. et al. Social network size affects neural circuits in macaques. Science 334, 697–700 (2011).
Dal Monte, O., Chu, C. C. J., Fagan, N. A. & Chang, S. W. C. Specialized medial prefrontal–amygdala coordination in other-regarding decision preference. Nat. Neurosci. 23, 565–574 (2020).
Grabenhorst, F., Baez-Mendoza, R., Genest, W., Deco, G. & Schultz, W. Primate amygdala neurons simulate decision processes of social partners. Cell 177, 986–998 (2019).
Allsop, S.A. et al. Corticoamygdala transfer of socially derived information gates observational learning. Cell 173, 1329–1342 (2018).
Li, D., Babcock, J. & Parkhurst, D. J. openEyes: a low-cost head-mounted eye-tracking solution. In Proc. 2006 symposium on Eye tracking research & applications 95–100 (ACM, 2006).
Mitz, A. R., Chacko, R. V., Putnam, P. T., Rudebeck, P. H. & Murray, E. A. Using pupil size and heart rate to infer affective states during behavioral neurophysiology and neuropsychology experiments. J. Neurosci. Methods 279, 1–12 (2017).
Meyers, E. M., Freedman, D. J., Kreiman, G., Miller, E. K. & Poggio, T. Dynamic population coding of category information in inferior temporal and prefrontal cortex. J. Neurophysiol. 100, 1407–1419 (2008).
We thank Y. Kfir for scientific and technical advice; E. Kahana and N. Samuel for medical and surgical procedures; D. Goldin for engineering design; and E. Furman-Haran and F. Attar for MRI procedures. This work was supported by ISF 2352/19 and ERC-2016-CoG 724910 grants to R. Paz.
The authors declare no competing interests.
Peer review information Nature thanks Jean-René Duhamel, Ziv M. Williams and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data figures and tables
a, Same format as in Fig. 1f but for the all-shutter ROI (and not just face ROI). As can be seen, the monkeys looked at the face and eyes ROI mainly in the interactions with the human intruder. Left, the gaze density during all the sessions. b, Same format as in Fig. 1g (left), but aligned to the first time the monkeys looked to the ROI of the intruder’s eyes in each trial separately. c, Same format as in Fig. 1g (right) separately for each monkey.
a, Examples of three original frames with different expressions, corresponding to the schematic in Fig. 1h. b, For every recording session, we averaged over all frames from the baseline period resulting in the mean image. Baseline was taken over the period before any trial when the monkey was alone in the room with a closed shutter. c, An example of a frame during the EC interaction. d, The mean frame (b) is subtracted from the frame in c during the interaction, to obtain a ‘diff’ (or delta) image. Three ROIs were defined manually for every day: upper face, ears and lower face. e, The r.m.s. of every ROI is calculated. Data are mean ± s.e.m. The differences between EC and NEC in the upper part of the face are shown. The other parts and ROIs are shown in Fig. 1. The black line represents a significant difference; P < 0.05, two-sided Student’s t-test; n = 1,480 trials in NEC and 1,628 trials EC).
a, Same format as in Fig. 3a, b. The population decoding accuracy is shown when training on eye gaze (NEC versus EC) and testing on valence (aversive versus appetitive), using CS-related activity. Data are mean ± s.d.; n = 1,000 bootstrap replicates, n = 203 amygdala neurons. b, Same as in a but using US-related activity.
a, If overall-activity drives the successful decoding in the US epoch, we expected to find an overall change in the firing rate (increase or decrease) for gaze and US valence. Indeed, we found that there were more valence-positive neurons (increased firing rate to in response to the air puff) in the amygdala in the US epoch, and that there were more gaze-positive neurons (increased firing rate in response to EC) in the amygdala. Inset graphs show mean ± s.e.m.; ***P < 1× 10−3; Z-test; n = 203 amygdala neurons, n = 356 ACC neurons. b, Decoding accuracy with and without neurons that encode gaze. Black and red lines represent the mean and median, respectively; n = 203 amygdala neurons, n = 1,000 bootstrap replicates.
a, Same format as in Fig. 4h–j. Population decoding accuracy for real and shuffled amygdala neurons. Black and red lines represent the mean and median, respectively; n = 203 amygdala neurons. b, Same as in a for ACC activity. n = 356 ACC neurons. c, Cumulative distribution of the difference in decoding accuracy between real and shuffled neurons. ***P < 1× 10−3; two-sample Kolmogorov–Smirnov test; n = 203 amygdala neurons.
a, An example of vocalizations during one HIT trial, measured using a microphone placed in close proximity to the monkey (Methods). Inset, the proportion of trials in which vocalizations occur. There was a very small proportion of trials in which vocalization occurred, and it was similar across EC and NEC trials. χ2 test; P = 0.88; n = 1,738 NEC trails and 1,807 EC trials. Owing to the low number of vocalizations, we were not able to characterize different types of vocalizations. In addition, we repeated analyses after removing trials during which vocalizations occur, and the main results were unchanged. b, An example of movement in one trial in response to the human intruder, measured using an accelerometer attached to the chair of the monkey (Methods). This also occurred during a small number of trials, and it was similar across EC and NEC trials. In addition, we repeated analyses after removing these trials, and the main results were unchanged. c, The overall change in facial expressions between EC and NEC (as in Fig. 1i). The r.m.s. of the change between the image over the whole face (main) and only for the lower half of the face (inset), compared to the neutral expression obtained from averaging over the baseline period when the monkey was alone (Methods). Data are mean ± s.e.m. There is a significant difference (P < 0.05); two-sided Student’s t-test; n = 1,703 NEC trials and 1,765 EC trials. d, Same as in c after applying the thinning method (iteratively selecting trials to obtain a similar distribution of behaviour across EC and NEC; Methods). We applied the same method also to eye movements. e, Decoding accuracy using only trials with similar behaviour across EC and NEC, taken after thinning as described in d. The results remain the same (compare to Fig. 4h). Red and black lines indicate the median and mean, respectively; n = 203 amygdala neurons; n = 1,000 bootstrap replicates.
a–d, Decoding accuracy was divided into the first and second half of trials and similar results were obtained. The presentation is a merged format of Fig. 4h, i. CS-related activity (a, b) or US-related activity (c, d) in the first half (a, c) and second half (b, d) of trials is shown. Red and black lines indicate the median and mean, respectively; n = 203 amygdala neurons, n = 356 ACC neurons; n = 1,000 bootstrap replicates.
a, Left, we divided the amygdala neurons into three groups: the first contains neurons that increased their firing rate (FR) in response to gaze and valence (61/203, positive β values in Fig. 4a); the second group decreased FR in response to both gaze and valence (65/203, negative β values in Fig. 4a); and the third group increased FR in response to one condition and decreased in response to the other (77/203). For the first two groups, the decoding accuracy of valence based on gaze (similar analysis as in Fig. 4h for CS-related activity) was significantly higher than chance, indicating that the overall result reported in the main text is based on both increases and decreases in FR. Right, same but for ACC neurons. Red and black lines indicate the median and mean, respectively. b, Amygdala neurons were sorted according to degree of modulation (magnitude of βgaze × βvalence). The decoding accuracy (mean) and its variance for increasing group size (that is, 10 neurons with highest modulation, 20 neurons… and so on) was recalculated. These values were compared to randomly chosen groups of similar size (green inset, notice the linear increase). The decoding accuracy increased until reaching a group size of 120–130 neurons; which is the number of neurons that contained the first two groups from a that either increased or decreased the firing rate, (but not mixed activity). Bottom, the proportion of neurons from the two groups. It can be seen that both groups contributed to the increased accuracy. These results further support the conclusion that the shared neural mechanisms are not due only to an increased firing rate, as an indication of saliency or alertness. n = 203 amygdala neurons.
a, We included monkey-intruder blocks (top) in a similar way to the HIT trials (bottom). The same neurons reported in the main analysis were recorded during the monkey–monkey interactions. For each recording session, on average two (out of six) monkeys served as intruders. All of the monkeys had lived together for several years. b, Neurons in the amygdala (n = 203), as well as in the ACC (n = 356), code for species and can differentiate between human and monkey intruders. Moreover, neurons differentiate between NEC human trials and monkey-intruder trials. Data are mean and s.e.m. c, In contrast to the findings in Fig. 4a, there was no significant correlation (Pearson’s correlation, r = 0.05, P = 0.45, n = 203) between βspecies and βCSvalence, strongly arguing against a correlated-selectivity mechanism between species and CS. d, Decoding accuracy of CS valence after training the decoder to differentiate species, was not different from chance level and significantly smaller than the decoding accuracy of CS valence based on gaze. Red and black lines indicate the median and mean, respectively. n = 203 amygdala neurons; n = 1,000 bootstrap replicates. e, Differences in heart-rate variability between monkey-intruder and NEC trials (as shown for EC and NEC trials (Fig. 1k)). *P < 0.05; two-sided Student’s t-test; n = 1,703, 1,765 and 1,620 trials in NEC, EC and monkey trials. f, Despite differences in heart-rate variability (e), the findings in d remained similar when using either only EC or only NEC trials of the human intruder (n = 203 amygdala neurons). Red and black lines indicate the median and mean, respectively.
a, We included neutral trials, in which the opening of the shutter (CS) is followed by nothing. b, The heart rate was significantly lower in neutral trials compared to all others types of trial and, specifically, it was lower than in NEC trials. Top left, difference in heart rate, same as in Fig. 1. j. Top right, difference in heart rate in the control days that included neutral trials, showing the same trend for all types of trial and no modulation for neutral trials. Together, these results indicate that the NEC trials were not salience-free, but rather highly salient in a different manner than the EC trials. n = 1,703, 1,765, 1,620, 1,352 and 712 trials for NEC, EC, monkey, reward and air puff trails, respectively. Data are mean ± s.e.m.; ***P < 1× 10−3; two-sided Student’s t-test.
About this article
Cite this article
Pryluk, R., Shohat, Y., Morozov, A. et al. Shared yet dissociable neural codes across eye gaze, valence and expectation. Nature 586, 95–100 (2020). https://doi.org/10.1038/s41586-020-2740-8