Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

The effect of face patch microstimulation on perception of faces and objects

Abstract

What is the range of stimuli encoded by face-selective regions of the brain? We asked how electrical microstimulation of face patches in macaque inferotemporal cortex affects perception of faces and objects. We found that microstimulation strongly distorted face percepts and that this effect depended on precise targeting to the center of face patches. While microstimulation had no effect on the percept of many non-face objects, it did affect the percept of some, including non-face objects whose shape is consistent with a face (for example, apples) as well as somewhat facelike abstract images (for example, cartoon houses). Microstimulation even perturbed the percept of certain objects that did not activate the stimulated face patch at all. Overall, these results indicate that representation of facial identity is localized to face patches, but activity in these patches can also affect perception of face-compatible non-face objects, including objects normally represented in other parts of inferotemporal cortex.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Experimental scheme.
Figure 2: Experiment 1: face patch stimulation exerts a large effect on perception of facial identity.
Figure 3: The dependence of effect magnitude on proximity to the center of a face patch.
Figure 4: Experiments 2 and 3: effect of stimulation outside the face patches on face perception and effect of stimulation inside face patch AM on perception of both general and round non-face objects.
Figure 5: Experiment 4a: effect of face patch stimulation on perception of abstract faces.
Figure 6: fMRI and electrophysiological responses to abstract Mooney stimuli.
Figure 7: Experiment 4b: effect of stimulation inside face patch AM on perception of non-face objects, part II.
Figure 8: Summary of results.

Similar content being viewed by others

References

  1. Gross, C.G., Rocha-Miranda, C.E. & Bender, D.B. Visual properties of neurons in inferotemporal cortex of the macaque. J. Neurophysiol. 35, 96–111 (1972).

    CAS  PubMed  Google Scholar 

  2. Tsao, D.Y., Freiwald, W.A., Tootell, R.B.H. & Livingstone, M.S. A cortical region consisting entirely of face-selective cells. Science 311, 670–674 (2006).

    CAS  PubMed  PubMed Central  Google Scholar 

  3. Freiwald, W.A. & Tsao, D.Y. Functional compartmentalization and viewpoint generalization within the macaque face-processing system. Science 330, 845–851 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  4. Issa, E.B. & DiCarlo, J.J. Precedence of the eye region in neural processing of faces. J. Neurosci. 32, 16666–16682 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  5. McMahon, D.B., Jones, A.P., Bondar, I.V. & Leopold, D.A. Face-selective neurons maintain consistent visual responses across months. Proc. Natl. Acad. Sci. USA 111, 8251–8256 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  6. Haxby, J.V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001).

    CAS  PubMed  Google Scholar 

  7. Hasson, U., Levy, I., Behrmann, M., Hendler, T. & Malach, R. Eccentricity bias as an organizing principle for human high-order object areas. Neuron 34, 479–490 (2002).

    CAS  PubMed  Google Scholar 

  8. Konkle, T. & Oliva, A. A real-world size organization of object responses in occipitotemporal cortex. Neuron 74, 1114–1124 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  9. Wiggett, A.J., Pritchard, I.C. & Downing, P.E. Animate and inanimate objects in human visual cortex: evidence for task-independent category effects. Neuropsychologia 47, 3111–3117 (2009).

    PubMed  Google Scholar 

  10. Haushofer, J., Baker, C.I., Livingstone, M.S. & Kanwisher, N. Privileged coding of convex shapes in human object-selective cortex. J. Neurophysiol. 100, 753–762 (2008).

    PubMed  PubMed Central  Google Scholar 

  11. Yue, X., Pourladian, I.S., Tootell, R.B. & Ungerleider, L.G. Curvature-processing network in macaque visual cortex. Proc. Natl. Acad. Sci. USA 111, E3467–E3475 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  12. Srihasam, K., Vincent, J.L. & Livingstone, M.S. Novel domain formation reveals proto-architecture in inferotemporal cortex. Nat. Neurosci. 17, 1776–1783 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  13. Verhoef, B.-E., Vogels, R. & Janssen, P. Contribution of inferior temporal and posterior parietal activity to three-dimensional shape perception. Curr. Biol. 20, 909–913 (2010).

    CAS  PubMed  Google Scholar 

  14. Salzman, C.D., Britten, K.H. & Newsome, W.T. Cortical microstimulation influences perceptual judgements of motion direction. Nature 346, 174–177 (1990).

    CAS  PubMed  Google Scholar 

  15. Histed, M.H., Ni, A.M. & Maunsell, J.H. Insights into cortical mechanisms of behavior from microstimulation experiments. Prog. Neurobiol. 103, 115–130 (2013).

    PubMed  Google Scholar 

  16. Afraz, S.-R., Kiani, R. & Esteky, H. Microstimulation of inferotemporal cortex influences face categorization. Nature 442, 692–695 (2006).

    CAS  PubMed  Google Scholar 

  17. Penfield, W. & Perot, P. The brain's record of auditory and visual experience. A final summary and discussion. Brain 86, 595–696 (1963).

    CAS  PubMed  Google Scholar 

  18. Allison, T. et al. Face recognition in human extrastriate cortex. J. Neurophysiol. 71, 821–825 (1994).

    CAS  PubMed  Google Scholar 

  19. Puce, A., Allison, T. & McCarthy, G. Electrophysiological studies of human face perception. III: Effects of top-down processing on face-specific potentials. Cereb. Cortex 9, 445–458 (1999).

    CAS  PubMed  Google Scholar 

  20. Parvizi, J. et al. Electrical stimulation of human fusiform face-selective regions distorts face perception. J. Neurosci. 32, 14915–14920 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Afraz, A., Boyden, E.S. & DiCarlo, J.J. Optogenetic and pharmacological suppression of spatial clusters of face neurons reveal their causal role in face gender discrimination. Proc. Natl. Acad. Sci. USA 112, 6730–6735 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  22. Fisher, R.A. On the interpretation of χ2 from contingency tables, and the calculation of P. J.R. Stat. Soc. 85, 87–94 (1922).

    Google Scholar 

  23. Grimaldi, P., Saleem, K.S. & Tsao, D. Anatomical connections of the functionally defined “face patches” in the macaque monkey. Neuron 90, 1325–1342 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Freiwald, W.A., Tsao, D.Y. & Livingstone, M.S. A face feature space in the macaque temporal lobe. Nat. Neurosci. 12, 1187–1196 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  25. Meyers, E.M., Borzello, M., Freiwald, W.A. & Tsao, D. Intelligent information loss: the coding of facial identity, head pose, and non-face information in the macaque face patch system. J. Neurosci. 35, 7069–7081 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  26. Mooney, C.M. Age in the development of closure ability in children. Can. J. Psychol. 11, 219–226 (1957).

    CAS  PubMed  Google Scholar 

  27. Taubert, J., Van Belle, G., Vanduffel, W., Rossion, B. & Vogels, R. Neural correlate of the thatcher face illusion in a monkey face-selective patch. J. Neurosci. 35, 9872–9878 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Jonas, J. et al. Focal electrical intracerebral stimulation of a face-sensitive area causes transient prosopagnosia. Neuroscience 222, 281–288 (2012).

    CAS  PubMed  Google Scholar 

  29. Moeller, S., Freiwald, W.A. & Tsao, D.Y. Patches with links: a unified system for processing faces in the macaque temporal lobe. Science 320, 1355–1359 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  30. Knoll, M. & Kugler, J. Subjective light pattern spectroscopy in the encephalographic frequency range. Nature 184 (Suppl. 23), 1823–1824 (1959).

    PubMed  Google Scholar 

  31. Brindley, G.S. & Lewin, W.S. The sensations produced by electrical stimulation of the visual cortex. J. Physiol. (Lond.) 196, 479–493 (1968).

    CAS  Google Scholar 

  32. Vignal, J.P., Chauvel, P. & Halgren, E. Localised face processing by the human prefrontal cortex: stimulation-evoked hallucinations of faces. Cogn. Neuropsychol. 17, 281–291 (2000).

    CAS  PubMed  Google Scholar 

  33. Fuster, J.M. & Jervey, J.P. Neuronal firing in the inferotemporal cortex of the monkey in a visual memory task. J. Neurosci. 2, 361–375 (1982).

    CAS  PubMed  PubMed Central  Google Scholar 

  34. Baylis, G.C. & Rolls, E.T. Responses of neurons in the inferior temporal cortex in short term and serial recognition memory tasks. Exp. Brain Res. 65, 614–622 (1987).

    CAS  PubMed  Google Scholar 

  35. Vogels, R. & Orban, G.A. Activity of inferior temporal neurons during orientation discrimination with successively presented gratings. J. Neurophysiol. 71, 1428–1451 (1994).

    CAS  PubMed  Google Scholar 

  36. Mikami, A. & Kubota, K. Inferotemporal neuron activities and color discrimination with delay. Brain Res. 182, 65–78 (1980).

    CAS  PubMed  Google Scholar 

  37. Miller, E.K., Li, L. & Desimone, R. Activity of neurons in anterior inferior temporal cortex during a short-term memory task. J. Neurosci. 13, 1460–1478 (1993).

    CAS  PubMed  PubMed Central  Google Scholar 

  38. Vogels, B. & Orban, G.A. Does practice in orientation discrimination lead to changes in the response properties of macaque inferior temporal neurons? Eur. J. Neurosci. 6, 1680–1690 (1994).

    CAS  PubMed  Google Scholar 

  39. Chelazzi, L., Miller, E.K., Duncan, J. & Desimone, R. A neural basis for visual search in inferior temporal cortex. Nature 363, 345–347 (1993).

    CAS  PubMed  Google Scholar 

  40. Ohayon, S., Freiwald, W.A. & Tsao, D.Y. What makes a cell face selective? The importance of contrast. Neuron 74, 567–581 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  41. Ohayon, S. & Tsao, D.Y. MR-guided stereotactic navigation. J. Neurosci. Methods 204, 389–397 (2012).

    PubMed  Google Scholar 

  42. Geusebroek, J.M., Burghouts, G.J. & Smeulders, A.W. The Amsterdam Library of Object Images. Int. J. Computer Vision 61, 103–112 (2005).

    Google Scholar 

  43. Stanislaw, H. & Todorov, N. Calculation of signal detection theory measures. Behav. Res. Methods Instrum. Comput. 31, 137–149 (1999).

    CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank N. Schweers for technical support, as well as N. Kanwisher, S. Kornblith, M. Livingstone, members of the Tsao lab and the three anonymous reviewers for critical comments. This work was supported by the Howard Hughes Medical Institute, the National Institutes of Health (1R01EY019702) and the Humboldt Foundation (to S.M.).

Author information

Authors and Affiliations

Authors

Contributions

S.M. and D.Y.T. designed the experiments, interpreted the data and wrote the paper. S.M. and D.Y.T. conducted all experiments except those shown in Figure 6, which were done by T.C. and L.C., and Supplementary Figure 8, which was done by L.C. S.M. analyzed the data.

Corresponding author

Correspondence to Doris Y Tsao.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Mean change in performance caused by microstimulation of each face patch.

Dark gray bars for same identity trials, light gray bars for different identity trials, error bars show the standard error of the mean. The results of individual sessions are shown as circles, filled circles show session in which the effect size was significant with p <= 0.01. As shown in Figure 3, effect size per session correlates with the face selectivity of each stimulation site, explaining the sessions with smaller effect sizes.

Supplementary Figure 2 The dependence of effect magnitude for faces on stimulation current strength.

(a, b) Behavioral performance on same- and different-identity trials, with and without microstimulation, for stimulation strength magnitude of 100 μA (a) and 300 μA (b), with stimulation performed in the same session (AM, M1). (c, d) Another session, comparing effect on behavior of face patch stimulation at 100 μA and 300 μA (AM, M1). (e, f) Effect of face patch stimulation at 100 μA and 200 μA during the same session (AM, M2). (g, h) Effect of face patch stimulation at 100 μA and 200 μA during the same session (AM, M2). (i, j, k) Effect of face patch stimulation at 100 μA, 200 μA, and 300 μA during the same session (ML, M2). Gray bars denote trials without, red bars trials with electrical micro-stimulation; Darker bars show the performance for same identity trials, lighter bars for different identity trials. *: P < 0.05; **: P < 0.01; ***: P < 0.005; Fisher’s exact test (see Supplementary Table 3 for exact P-values). The lower plot in each panel shows the change in percentage points caused by electrical stimulation for same identity trials in dark gray and for the different identity trials in light gray.

Supplementary Figure 3 The dependence of effect magnitude for round objects on stimulation current strength and electrode position, for subject M1 for three different sessions.

(a) Low stimulation current (50 μA) inside face patch AM. (b) Intermediate stimulation current (100 μA) inside face patch AM. (c) Large stimulation current (200 μA) outside the face patches. Top row: Behavioral performance on same and different trials, with and without microstimulation, for stimulation in face patch AM of monkey M1, for faces, apples, citrus fruits, teapots, and clocks. 2nd row: Size of stimulation effect in percentage points for the same five categories. 3rd row: Behavioral performance on same and different identity trials of the face identification task of the same session. 4th row: Size of stimulation effect for the face identification task. Conventions as in Suppl. Fig. 2. *: P < 0.05; **: P < 0.01; ***: P < 0.005; Fisher’s exact test (see Supplementary Table 3 for exact P-values).

Supplementary Figure 4 The dependence of effect magnitude on proximity of the stimulation site to the center of the face patch, for the five different object categories from Experiment 3.

Same conventions as in Figure 3a. Pooled data for 14 sessions (M1: 9, M2: 5) showing how the magnitude of electrical microstimulation effect on same (left column) and different identity trials (right column) correlates with the face selectivity of the target location as measured by fMRI. Out of the five object categories (faces (a), apples (b), citrus fruit (c), pots (d) and clocks (e)), only for faces was there a significant correlation (same identity trials: p: 0.044706, correlation coefficient r = -0.54319, r2 = 0.2951; different identity trials: p: 0.0053008, correlation coefficient r = 0.70014, r2 = 0.4902).

Supplementary Figure 5 Experiment 2: effect of stimulation inside face patch AM on the perception of nonround objects.

Conventions as in Fig. 2. (a) Effect of AM stimulation on non-round object perception in monkey M1. Performance was significantly worse on microstimulation “same” trials (reduction by 7.77 percentage points). (c) Performance change in percentage points caused by electrical microstimulation for same- (dark gray bar) and different-identity trials (light gray bar) (b) Same as (a) and (d) same as (c), for stimulation in monkey M2. Stimulation current was 300 μA. *: P < 0.05; Fisher’s exact test (see Supplementary Table 3 for exact P-values).

Supplementary Figure 6 Experiment 4b: effect of stimulation inside face patch AM on perception of non-face objects II in M2.

(a) Effect of face patch stimulation on perception of house line drawings, house cartoons, house silhouettes, Mooney faces, and upside down Mooney faces. (b) Effect obtained in same experimental session, for face stimuli of Experiment 1. (c) Performance change in percentage points caused by electrical microstimulation for same- (dark gray bar) and different-identity trials (light gray bar) for each of the five categories of non-face objects II, and (d) for the faces of Experiment 1. Stimulation current was 200 μA. Conventions as in Fig. 4. *: P < 0.05; **: P < 0.01; ***: P < 0.005; Fisher’s exact test (see Supplementary Table 3 for exact P-values).

Supplementary Figure 7 Electrode position for M1 showing that, for the data presented in Figure 7, the electrode tip was located inside face patch AM.

Scale bar (top left) = 1 cm.

Supplementary Figure 8 Activation to faces, cartoon houses, real houses and real objects.

(a) Coronal slices showing fMRI activation to the contrast faces > objects. Face patches are indicated by arrows. (b, c) Slices from the same animal as (a), showing activation to the contrast cartoon houses > objects (b), and real houses > objects (c). (d) Flat maps of the left and right hemisphere visual cortex, showing the same data as (a-c). Face patches are indicated by green outlines. Notice overlap between face patches ML and PL and cartoon house activation (middle two panels), which is absent for real house activation (right two panels). Face patches indicated by green outlines in middle and right panels. (e) Beta values for faces, cartoon houses, real houses, and real objects, from PL/ML, showing strong activation to faces and to cartoon houses. Box-and-whisker plot indicates the median value (red line), the 25–75th percentiles (box) and +/- 2.7 sigma (whiskers).

Supplementary Figure 9 Experiment 5: dependence of effect magnitude on stimulation timing.

Behavioral performance for same- and different-identity trials, across four different stimulation conditions: no stimulation (gray bars), stimulation during cue 1 and cue 2 (yellow bars), stimulation during cue 1 (green bars), stimulation during cue 2 (red bars). Plots on the right show the change in percentage points caused by electrical stimulation (electrical stimulation during cue1: green; during cue2: red; during both cues: yellow). Darker bars denote same-identity trials, lighter bars different-identity trials. *: P < 0.05; **: P < 0.01; ***: P < 0.005; Fisher’s exact test (see Supplementary Table 3 for exact P-values). (a): ML, M1, (b): AM M1, (c): AM M2. (d): AM M1: Stimulation experiment in M1, with stimulation trains of 50 ms duration (images were presented for 200 ms each), instead of 200 ms (b). All trains were delayed by 75 ms relative to visual stimulation to account for the typical response latency of anterior temporal cortex.

Supplementary Figure 10 Analysis of data in Supplementary Figure 9, separating data for each of 32 different cue 2 identities.

(a, b) Analysis of data in panels (a), (b) of Supplementary Figure 9, respectively. Numbers below each bar indicate number of contributing trials, *: P < 0.05; **: P < 0.01; ***: P < 0.005; Fisher’s exact test (see Supplementary Table 5 for exact P-values). A 3-way ANOVA examining main effects and interactions of identity, trial type (same/different) and microstimulation (yes/no) showed significant main effects for trial (same/different) and stimulation condition, but not for identity, and a significant interaction between trial and stimulation. The results do not support the idea that effect of stimulation depended on the specific identity of the face: this would show up as a two-way interaction between stimulation and identity or three-way interaction between stimulation, trial type, and identity. M1 (a), trial type F(1, 92) = 401.58, p < 0.0001; stimulation F(3, 92) = 110.74, p < 0.0001; identity F(31, 92) = 1.51, p = 0.0666; trial*stimulation F(3, 92) = 39.75, p < 0.0001; trial*identity F(31, 92) = 1.51, p = 0.0665, stimulation*identity F(93, 92) = 0.94, p = 0.6137; M2 (b), trial type F(1, 92) = 150.51, p < 0.0001; stimulation F(3, 92) = 57.88, p < 0.0001; identity F(31, 92) = 1.15, p = 0.2937; trial*stimulation F(3, 92) = 10.22, p < 0.0001; trial*identity F(31, 92) = 1.16, p = 0.2846, stimulation*identity F(93, 92) = 0.83, p = 0.8079.

Supplementary Figure 11 Behavioral performance from training sessions and from experimental sessions from nonmicrostimulated trials.

This figure shows the task performance over the course of multiple sessions, always starting with the first session a stimulus set was introduced. Performance was averaged over same and different identity trials. (a) M1: The red line shows the initial task that consisted of performing same/different discrimination between five images of grape clusters and five images of single tomatoes; for the same condition we always presented the exact same image, for different we always selected one image from the five exemplars of the other category. The green line shows the initial performance on the initial face identity discrimination task. In this task we presented color photographs of 5 persons at 5 different views each, for the same condition we only selected the 2nd cue image from the 4 images of each identity not used as 1st cue, for the different condition we randomly selected one of the 20 images belonging to the remaining identities. Note how the performance for grapes and tomatoes ramps up over 4 sessions and how well the animal transferred to the face identification task (the animal had discriminated different face stimuli in past experiments). (b) Example stimuli. Left: all grape and tomato images used in initial training. Right: three exemplary identities at five views. (c) M1 and (d) M2 performance in >40 sessions with the 32 identities at 6 different facial expressions task (see Experiment 1). Note with M2 there was an initial ramping up from near chance performance, but both animals generalized quickly from the initially trained face set to the new set. (e & f) Performance in the different versions of the object identification task (as described in Experiment 2); both M1 and M2 immediately generalized to the different object sets. Note that we only added more stimuli, so the 16 objects set was a subset of both the 19 and 28 objects sets. (g & h) Performance for Experiment 3: both animals immediately generalized to the five new categories. Note that on the 8th session, M1 changed his strategy to guessing only to return to “normal” performance on the next session. (i & j) Performance on the abstract face and house stimuli from Experiment 4b; M1 immediately generalized to the new stimuli, while M2’s performance improved over the first few sessions before stabilizing.

Supplementary Figure 12 Neuronal response selectivity at the different stimulation sites, across animals.

The upper part of each panel shows the normalized mean response over 200 ms for each unit to each of the 96 consensus images, while the middle part shows the population mean response. Note that most units were recorded for the minimal time to see whether there was a difference between responses to face and non-face categories, so often, images were not repeated enough times to allow the normalization to work; such occurrences are colored black in the upper panels and ignored for the averaging shown in the middle panels. The lower part of each panel shows the distribution of the face selectivity indices for all units per patch and animal, distance from the midpoint at zero correlates with face selectivity. The category of each image is color-coded by the rainbow bar. The order of the presented population data mainly follows Fig. 2 to facilitate direct comparison: (a) AM of monkey M1 (b) AM of M2, (c) ML of M1, (d) ML of M2, (e) AL of M1, (f) MF of M2, (g) AF of M1 (h) control site outside of the face patches in M1.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–12 and Supplementary Tables 1–4 (PDF 4116 kb)

Supplementary Methods Checklist (PDF 472 kb)

Supplementary Table 5

Selected statistics for Supplementary Figure 10. (XLSX 20 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moeller, S., Crapse, T., Chang, L. et al. The effect of face patch microstimulation on perception of faces and objects. Nat Neurosci 20, 743–752 (2017). https://doi.org/10.1038/nn.4527

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.4527

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing