Main

Poor social adaptation and functioning is a hallmark of schizophrenia and may be related to the deficits at different levels of cognition that have been documented in the condition (Green 1996; Dickerson et al. 1996) . Numerous reports over the last forty years (reviewed in Hellewell and Whittaker 1998) indicate that patients with schizophrenia have a deficit in properly recognizing facial affect—the emotional value aspects of facial expressions (Borod et al. 1993; Streit et al. 1997; Addington and Addington 1998; Federman et al. 1998; Habel et al. 2000; Izard 1959; Dougherty et al. 1974; Muzekari and Bates 1977; Walker et al. 1980; Heimberg et al. 1992; Iscoe and Veldman 1963; Pillowski and Bassett 1980). Yet the neural correlates of these deficits have not been established. Facial affect recognition is instrumental in social learning and adaptation. Recent functional neuro-imaging preliminary reports have characterized abnormalities in relative blood flow changes—a likely measure of brain activity changes (Heeger et al. 2000; Rees et al. 2000)—in specific cerebral regions of schizophrenic patients while they performed working memory or discrimination tasks involving facial affect processing (Quintana and Kovalik 1999; Hariri et al. 1999). It is difficult to determine whether changes in brain activity observed in schizophrenia represent primary deficits or secondary—even compensatory—phenomena, being that this is a chronic condition where even treatment effects have been implicated in the development of brain abnormalities. However, it is plausible to believe that, if schizophrenic patients exhibit activity increases in specific brain areas—when compared to controls—while correctly performing tasks supported by those areas, these activity increases may be the result of, or represent, cerebral compensatory efforts.

In primates, the observation of actions and gestures performed by others activates the internal representation of these actions, eliciting patterns of neuronal activity similar to those seen when the observing individual executes the same actions (Gallese et al. 1996; Hari et al. 1998; Iacoboni et al. 1999; Rizzolatti et al. 1999; Nishitani and Hari 2000). “Resonant” neural systems that match observation and execution of actions have been described in the premotor and motor cortical areas of human and non-human primates (Hari et al. 1998; Iacoboni et al. 1999; Rizzolatti et al. 1999; Nishitani and Hari 2000). Single-unit activity occurring both during behavior observation and action, especially related to hand and mouth gestures, has been recorded from neurons in the premotor cortex of macaque monkeys that supports the operation of those systems, and it has been termed mirror activity (Gallese et al. 1996; Rizzolatti et al. 1999). Resonant or mirroring systems are believed to have a role in recognizing and interpreting actions or gestures performed by others and their meanings (Gallese et al. 1996; Hari et al. 1998; Iacoboni et al. 1999; Rizzolatti et al. 1999; Nishitani and Hari 2000; Rizzolatti and Arbib 1998; Gallese and Goldman 1998; Gallese and Goldman 1998), as well as in motor rehearsal (Decety et al. 1994), imitation (Iacoboni et al. 1999; Nishitani and Hari 2000), and learning (Decety et al. 1994). Thus, they support social adaptation and functioning.

We hypothesized that the correct processing of facial affect during working memory (WM) by schizophrenic patients would invoke compensatory brain mechanisms including increased activity in cerebral areas that participate in the internal representation of facial expressions for the purpose of understanding their meaning. We further hypothesized that, because of the relationship between facial affect recognition and social learning and adaptation, and the known deficits in the latter in schizophrenia, the areas involved in those particular compensatory processes should be part of brain mechanisms known to participate in social learning. Thus, if our hypotheses were correct we should detect correlates of increased neuronal activation in schizophrenic patients—relative to controls—during correct performance of WM tasks requiring facial affect processing, particularly in brain areas related to facial affect representation and that also support social learning. An ideal candidate region for possessing those characteristics is the mirror system for facial expressions encompassing premotor—and possibly also motor—areas that control and coordinate facial movements for the construction of facial expressions.

To test our hypothesis, we used functional magnetic resonance imaging (fMRI) to measure localized relative blood flow changes—a correlate of brain activity (Heeger et al. 2000; Rees et al. 2000)—in premotor and motor areas of schizophrenic patients and healthy controls. The subjects performed simple visuo-motor task paradigms where a visual cue—a color circle or a diagram of a facial expression—indicated the correct choice between, respectively, two color circles or two facial diagrams presented a few seconds later with different degrees (50% or 100%) of probability (see Methods section). The task rules emphasized the retention of cue information (i.e., when the correct choice could not be anticipated) or the anticipation of responses (i.e., when it could). This task design has proven useful for understanding the interactions between separate cerebral areas of distributed networks in monkeys (Quintana and Fuster 1992, 1993, 1999).

METHODS

Subjects

We recruited eight volunteers (six males, two females, mean age: 29.25 ± 5.13, all right-handed) and eight schizophrenic patients (six males, two females, mean age: 35.22 ± 10.69, all right-handed) whose diagnosis had been established according to DSM-IV criteria (American Psychiatric Association). The subjects received a detailed explanation of the nature and possible consequences of the study and were entered in the protocol, approved by the UCLA School of Medicine and the West Los Angeles VA Health Care Center Institutional Review Boards in compliance with the Declaration of Helsinki and the guidelines from the National Institutes of Health, after their informed consent was obtained. The control subjects had no personal or familial history of neurological or psychiatric disorders. The schizophrenic subjects were all chronic (average length of illness 8.5 years), stable outpatients, medicated with modern (i.e., atypical) antipsychotics at standard oral doses (Olanzapine 10–20 mg/day, Risperidone 2–6 mg/day) known to have minimal effect on motor systems. None of the patients received concurrent anticholinergic medications. They were free of neurological conditions or symptoms, as revealed by a review of their past and current medical records.

Experimental Task Design

Functional MRI data were acquired while the subjects in both groups performed a series of block-design paradigms counterbalanced for order across subjects. For each paradigm, three resting blocks (black visual field) of approximately 24 seconds each were interleaved with two series of six trials (see Figure 1 ). Each trial consisted of the display, for 0.5 seconds, of a cue (see below) in the center of a black screen, a 7-sec delay (black screen only), and a choice between two stimuli similar to the cue, presented side-by-side in the middle level of a black screen background for up to 2.5 seconds. Once a choice was made according to the task contingencies explained below, the choice stimuli disappeared and the screen remained black for the rest of the 2.5-sec period, right after which the cue for the next trial appeared. Each block of six trials lasted approximately 70 seconds, of which 3.5 seconds (5% of total) corresponded to cue presentations, 42 seconds (60% of total) to delay periods, and, on the average, 7–10 seconds (10–15% of total) to choice execution and 14–17 seconds (20–25% of total) to intertrial periods. Thus, our design maximized the weight of delay activity in the task-related signal changes measured by fMRI. Overall, for the purpose of this study, each subject underwent four runs, each containing 12 trials.

Figure 1
figure 1

Temporal structure of an ex- perimental run and trial (see Methods section). R (1,2,3) = resting periods; A (1,2) = activation periods; s = seconds. Each activation period contained six task trials, whose temporal structure is also depicted in a lower small diagram. Top graph: time course of changes in signal intensity throughout an entire scanning run. Observe the prominent activation during the two activation periods of the run. Bottom graph: time course of changes in signal intensity during periods of activation (average of two activation periods of a run). Observe that the relative increases in signal intensity coincide with the delay periods of the six trials. Data corresponds to left Brodmann's area 4, patients group, anticipatory task performance with facial stimuli.

Full size image

We used two types of stimuli as cues: (a) colored circles, or (b) line drawings of faces depicting either a happy or a sad expression. The choices at the end of each trial were between two colored circles or two face diagrams presented side-by-side on a black background, respectively. For each of those two stimulus types, colors or faces, we designed two sets of cues. One set indicated with certainty the location—right or left—of the correct choice, whereas the other did not, all according to the following contingencies: a red or green cue, or a smiling or sad face, preceded a choice between red and green color circles or between smiling and sad faces respectively, presented side-by-side in random position over a black background after a 7-sec delay. The subjects were instructed to choose the response stimulus—right or left—that matched the cue. Thus, in the case of this first set of cues, the predictability of the correct response side was 50% (chance level). Alternatively (second set of cues), a blue or a yellow cue preceded a choice between two side-by-side white circles. And a smiling or a sad face, whose nose pointed at the right or left of the subject's visual field, preceded a choice between a smiling and a sad face (also presented side-by-side, the one on the left with its nose pointing to the left of the subject, the one on the right with a nose pointing to the right). The subjects had been instructed before the scanning session that after a blue color, the correct response was always the left white circle, and after a yellow color, the right white circle. They had also been instructed that the direction where the nose of the face cue was pointing always determined the correct response between the two face choices, left or right. Hence, the predictability of correct response here was 100%, although the subjects could not execute a response until the two choices appeared following the cue. Each trial series randomly used two cues of the same type (color or face) and of the same set (i.e., same level of response predictability—certain or uncertain correct choice side) (e.g., blue and yellow, red and green, smiling and sad faces, faces with right- and left-pointing noses).

Imaging Protocols

Subjects were fitted with magnetic-compatible goggles with built-in LCD screens on which the computer-programmed stimulation paradigms were displayed. They responded via button-pressing (left or right) on a mouse device with minimal movement of two fingers of the dominant (right) hand. Performance data was recorded by means of microcomputer control and statistically analyzed using a logistic regression model for response correctness and a multiple regression model for reaction time. Images were acquired using a GE 3T scanner with an echo-planar imaging upgrade. First, we obtained co-planar structural images consisting of 26 slices (4 mm thick, 1 mm gap) that covered the entire brain volume (TR/TE=4000/54 msec). Then we acquired gradient echo (GRE) functional images covering 14 slices (4 mm thick, 1 mm gap), beginning near the middle of the temporal lobes and moving upward (TR/TE=2000/45 msec). Because of this volume limitation, areas such as amygdala and cerebellum that may play a role in processing affect and memory were not explored in this study. For each subject, images were first realigned to correct for head motion (Woods et al. 1998), then normalized into a standard stereotaxic space. We smoothed the imaging data using a Gaussian filter set at 6 mm full-width at half-maximum to minimize noise and residual differences in gyral anatomy. Finally, a group analysis of the pre-processed images was completed using a general linear model with a delayed box-car reference function (SPM 99) (Friston et al. 1994), comparing average signal intensity during trial-performance periods with the average signal intensity during resting periods immediately before and after each series of trials. Additionally, we performed a first-level conjunction analysis (Friston et al. 1999) with implicit modeling of resting periods, contrasting the activations detected in both groups. Because of the described effects of motion on individual and group data (Hajnal et al. 1994; Friston et al. 1996), we carefully investigated the presence of motion at the individual and group level and concluded that only an insignificant drift of less than 0.4 mm, progressive and unrelated to task periods, was present in our samples. When a significant activation in the group data was identified, we investigated its variability in voxel intensity, cluster extent, and time-course across all subjects within our design matrix, in which inter-subject variability is normalized via proportional scaling. We found the variance in voxels of interest to be smooth across subjects and with few outliers, denoting a tight distribution of voxel intensity as it appears after scaling to whole brain mean signal of the group.

RESULTS

Both subject groups in our study, patients and controls, performed the tasks at statistically similar levels of accuracy, as determined by logistic (for response correctness, p = .09) and multiple (for reaction time, p = .49) regression model analyses of the performance data. Given the block paradigm design used, we could not—unfortunately—analyze activation patterns during trials with incorrect response, but the overall error rate was only 13.8 %. The imaging data analyses revealed discrete, statistically significant, relative blood flow changes in areas of the motor and premotor cortex, as well as in Broca's area and subcortical nuclei (Figure 2 , Table 1). Time series analyses confirmed that the relative fMRI signal intensity changes measured were related to task performance (Figures 1 and 2). We observed differences in activation patterns within and between groups mostly as a result of task variations in type of information (i.e., color circles or facial diagrams depicting an emotion). Overall, among both patients and controls, more activation in cortical motor systems was seen when the information needed for correct task performance was of emotional (i.e., facial expressions) than when it was of logical (i.e., colors) nature. Specific differences between schizophrenic patients and control subjects supporting our hypothesis occurred at the levels of premotor and motor cortex (in areas extending over the face, mouth, and lips representation), supplementary motor area (SMA), and Broca's area (Figure 2, Table 1).

Figure 2
figure 2

Cortical and subcortical areas showing statistically significant signal intensity increases related to task performance as determined by group analyses (see Methods section for details). Functional data are overlaid on Talairach-compatible templates from a brain MR atlas (Woods et al. 1998). Red rings circle significant activations, shown in yellow for the group of schizophrenic subjects and in blue for the control group (the same color code is used on top of a time series diagram indicating whether the graph represents data from patients or controls). A diagram on the top left corner represents the task trial structure, using facial stimuli as example. The figure is divided in four sectors. Activations displayed to the left of the vertical line are those seen during performance with facial expressions as stimuli, and to the right of the vertical line are those seen during color task performance. The horizontal line separates activations measured during mnemonic from those measured during anticipatory performance. Labels describe Brodmann's areas (BA = Brodmann's area; B = bilateral; L = left; R = right; Thal = thalamus; Put = Putamen; SMA = supplementary motor area). See Table 1 for anatomical description, Talairach coordinates of each activation's peak, cluster extents and statistical significance scores. For some activations of special relevance to the overall results, the corresponding time series of the signal variations of the most significant voxel over the task activation periods is displayed as an average of all runs from all subjects of the particular group and task (diagram insert). Periodic blue bands on the time series graphs represent delay (memory) periods of the task. In all of the time series the activity oscillates periodically six times, the exact number of trials in each activation period and thus of averaged delay periods represented in each graph. The relative position of the bands is precise with respect to the activity changes, because the activation paradigm design automatically compensated for any response time differences between trials or subjects, by adding a variable waiting period until 2.5 seconds after the end of the delay and the onset of the response period.

Full size image
Table 1 Performance-related Activations in Patients and Controls According to Task Demand and Stimulus Type
Full size table

During task performance requiring the processing of facial expressions depicted by a diagram, either by retaining facial information in WM or by anticipating responses on the basis of such information, schizophrenic subjects showed a pattern of bilateral increases in relative blood flow in Brodmann's premotor area 6, and unilateral (left, dominant) increases in area 4 and Broca's area (44 of Brodmann's). They also showed relative blood flow increases in SMA (bilaterally), thalamus (bilaterally, anterior and mediodorsal nuclei) and in left putamen during mnemonic task performance with facial expressions as cues, and left SMA and right Brodmann's area 44 during the anticipatory task performance with facial diagrams. On the other hand, also when using these diagrams as cues for task performance, healthy controls showed only premotor (Brodmann's area 6) activation in the left hemisphere during the mnemonic task and predominantly in the right premotor cortex during the anticipatory task, with a minimal activation also on the left side in this latter case (Figure 2, Table 1). Finally, the control group showed activation of area 47 of Brodmann's (inferior frontal cortex, IFC) that was absent in the patients group.

When the information needed to perform the task was color, however, the activation pattern differences between the two subject groups was no longer observed. We found limited activation of the left SMA—in both groups during retention of color information and only among the schizophrenic patients during anticipation of responses based on color information. In this later case, a significant activation of the left Broca's area, as well as of Brodmann's area 47 in the IFC, were seen solely in the control group (Figure 2, Table 1). Neither group showed area 47 activation in any other performance situation using color cues.

DISCUSSION

Our findings indicate that, during correct performance of decision-making tasks that demand processing facial expressions in working memory, motor cortical areas involved in the representation and execution of facial movements responsible for those expressions become significantly activated in patients with schizophrenia but not in control subjects. At the same time, areas of the IFC connected to limbic structures appear activated in normal controls only. These differences occur in spite of changes on the temporal dynamics of the tasks (i.e., remembering information or anticipating responses) as long as the information involves facial emotional expressions.

Two questions at least come to mind when interpreting our results: could they be the product of artifacts (e.g., subject's motion)? Can they be considered as indication of abnormal activation, perhaps compensatory, of a mirror system for facial expressions in schizophrenia? Regarding the first issue, both groups of subjects were required to perform the tasks under similarly strict motionless conditions, and asked to respond in an equally controlled manner. No significantly increased head motion artifacts, which could result from facial movements, were observed among the schizophrenic patients or detected during data analyses. The subjects were observed during the initial brief period of acquaintance with the task structure and rules, and during the brief practice they were allowed before their placement on the scanner bed; no particular facial movement was observed that could be responsible for the activation seen in the results. We did not obtain electromyographic records from the subjects during performance, a point that we could address in future studies in spite of technical difficulties. However, the fact that motor and premotor cortex activation in our study was seen only during the task performance guided by facial expression information further suggests that the phenomenon is not a reflection of subjects’ movements. The same argument applies to any putative medication effects on the results.

Correlating motor or premotor cortical activity detected with fMRI to any particular body structure is imprecise in the absence of corroborating evidence from concurrent movements. Although we did not obtain specific motor cortical maps for any of our subjects, the tridimensional coordinates of the activation areas here described do not disagree with those reported for the face, mouth and lips areas in somatotopic studies of the motor-premotor body representation (Grafton et al. 1991; Lotze et al. 2000), particularly when accounting for factors such as overlapping in cortical maps and the impact of averaging described in these and other studies (Grafton et al. 1993). Furthermore, the crucial information necessary to correctly perform our task was located on the mouth-lips area of the face diagrams used as cues. Thus, the observed changes in task-related signal intensity occurred during observation of a diagram depicting an action or gesture, and in cortical areas where such action or gesture is represented for a putative motor output (i.e., there is an implicit matching between observation and execution). Whereas, strictly speaking, we should not equal those changes to mirror activity, for a canonical definition of mirror activity requires a complete match between an action observation and its actual execution, we can interpret them as enhanced internal representation of the observed facial expressions. Another suggested interpretation is the presence of positive symptoms in the schizophrenic subjects. However, we did not find parallel differential activations in somatosensory cortex (which is an important component of the neuronal circuitry thought to be dedicated to the internal representation of emotions in the theory of mind), nor in visual cortex (which has been shown to be involved during active production of visual hallucinations and visual imagery). Finally, we did not detect statistical differences in performance levels between the groups or in activation of the visual cortex that would indicate an effect of cue complexity in the results.

In the case of the observed primary motor cortex activation, it is conceivable that an expansion of premotor activity might facilitate changes in corresponding primary motor areas. However, the co-activation of area 4, contralateral to the performing hand, and area 6 bilaterally has been reported in studies of execution-simulation matching (Roth et al. 1996). Moreover, the activation of Broca's area has been suggested as a crucial element in imitation and mirror systems in humans (Iacoboni et al. 1999). Hence, a selective co-activation of the representation of facial movements on these three cortical elements (i.e., left area 4, bilateral area 6, and Broca's area) in the group of patients and only during processing of information conveyed by observed facial gestures strongly suggests that an enhanced internal representation of the motor aspects of facial expressions helps subjects with schizophrenia to process the emotional meaning of these expressions. The lack of significant differences in overall performance or reaction time between controls and patients supports our hypothesis of a compensatory mechanism. We cannot directly correlate the abnormalities detected to any behavioral deficits in facial affect discrimination or processing in our subjects, who nonetheless exhibited concurrent activation deficits in IFC. However, we feel that our results can be interpreted in the context of the well-established presence of deficits in facial information processing in schizophrenic patients at large. Although such processing has been shown to involve the right somatosensory and parietal cortex (Haxby et al. 2000), the laterality seen in our results reflects the pattern described for mirror systems, and thus must be interpreted in the context of these systems.

Single-unit recording studies in monkeys and functional imaging studies in healthy human volunteers suggest that the motor systems can show activation in the absence of motor output, similar to that found during actual motor acts (Gallese et al. 1996; Hari et al. 1998; Iacoboni et al. 1999; Nishitani and Hari 2000; Roth et al. 1996; Jeannerod and Frak 1999). There is evidence that imaging a movement is associated with activation of the motor areas that would be normally responsible for generating such movement (Jeannerod and Frak 1999; Schnitzler et al. 1997). Resonant or mirroring systems that match execution and observation of an action have been described that may help in the understanding of the behaviors, actions, and intentions of other individuals in society (Gallese et al. 1996; Hari et al. 1998; Iacoboni et al. 1999; Rizzolatti et al. 1999; Nishitani and Hari 2000; Rizzolatti and Arbib 1998; Gallese and Goldman 1998; Decety et al. 1994). These systems are capable of representing “motor ideas” such as facial movements that convey emotional or intentional states of importance in social communication (Fadiga et al. 2000) and thus appear crucial for social functioning and adaptation, deficits of which are well known in schizophrenia.

Our main objective was to characterize abnormal brain mechanisms related to the well known social functioning deficits in schizophrenia, and that link social learning and adaptation with facial affect processing at the cerebral level. In our tasks, we wanted to emphasize the use—in WM—of information relevant for facial affect processing, because the known WM deficits in schizophrenic patients (Jeste et al. 1996) might enhance related compensatory activity under conditions of correct performance. We thus expected to see abnormal activity in areas involved in resonance or mirroring phenomena rather than in sensory areas related to instant (i.e., perceptual, non-mnemonic) discrimination of facial features such as the fusiform gyrus (Halgren et al. 2000) or the superior temporal sulcus or inferior frontal cortex (Nakamura et al. 1999; Haxby et al. 2000; Allison et al. 2000), which are the focus of other studies. A functional dissociation between areas involved in facial information processing has been suggested before when memory-encoding is involved (Decety et al. 1997). Characterizing compensatory mechanisms and their potential modulation is important for the social rehabilitation of schizophrenic patients. But it is well known that schizophrenia involves frontal function deficits (Jeste et al. 1996; Ingvar and Franzen 1974; Weinberger et al. 1986; Callicott et al. 1998; Heckers et al. 1999); in fact, prefrontal and inferior frontal abnormalities we will report elsewhere co-occurred in our patient's sample. What then is the relationship between the mechanisms described here and the prefrontal abnormalities observed in schizophrenia?

In normal individuals, the prefrontal cortex hosts mechanisms supporting cognitive processing and motor control, and thus the planning and execution of goal oriented behaviors in response to visual information (Fuster 1989). Working memory, attention, planning, and other executive prefrontal functions are crucial in preparing for and performing willed responses. Ultimately, however, the prefrontal cortex exercises its executive role by means of its control on motor systems activity and output. The prefrontal inhibition of unwanted, inappropriate, or untimely motor behavior is specially important for the unobstructed generation of adaptive behavior, and prefrontal gating mechanisms for perceptual inputs and motor outputs are well established (Fuster 1989; Hauser 1999; Konishi et al. 1999; Knight et al. 1999; Weinberger et al. 1983; Andreasen et al. 1986). Frontal lobe patients are known to produce uncontrolled behaviors imitating simple actions that other individuals perform in their presence (L'Hermitte et al. 1972). Frontal lobe pathology has been documented in schizophrenia (Jeste et al. 1996; Weinberger et al. 1986), and flawed frontal gating mechanisms have been hypothetically associated to the perceptual and information-processing deficits seen in this condition (Braff et al. 1995; Nuechterlein and Dawson 1984). Schizophrenic patients show motor abnormalities such as grimacing, echopraxia, posturing, stereotypy, echolalia and others in speech production, which could be the result of prefrontal gating failures. Hence, as these lines of evidence point out, it is likely that a prefrontal inhibitory deficit of motor system activity exists in schizophrenic patients. The mirror–like premotor and motor increased compensatory activity we describe here could be facilitated by such a prefrontal deficit, particularly when processing of facial affect information relevant for social functioning is required that demands additional resources. Patients with schizophrenia exhibit a number of sensory and perceptual processing deficits (Braff et al. 1995; Nuechterlein and Dawson 1984) likely contributing to prefrontal-based perception-action cycle abnormalities. Among those, deficits specifically related to facial affect discrimination may involve cerebral sensory systems such as the superior temporal sulcus, the fusiform gyrus, and the inferior frontal cortex, as some studies indicate (Hariri et al. 1999; Nakamura et al. 1999; Haxby et al. 2000; Allison et al. 2000). Yet the specific association of increased mirror-like representational mechanisms described in our patients with cues of obvious social value stresses the relationship between prefrontal deficits and poor social adaptation in schizophrenia. The fact that patients with schizophrenia appear to use relatively unnatural—when compared to normal individuals—brain mechanisms to interpret and properly use cues instrumental for social functioning may contribute to understanding the difficulties that these individuals feel and manifest in social situations.