People adapt with remarkable flexibility to reversal of the visual field caused by prism spectacles1,2. With sufficient time, this adaptation restores visually guided behaviour and perceptual harmony between the visible and tactile worlds1,2,3. Although it has been suggested that seeing one's own body is crucial for adaptation1,2, the underlying mechanisms are unclear. Here we show that a new representation of visuomotor mapping with respect to the hands emerges in a month during adaptation to reversed vision. The subjects become bi-perceptual3,4,5, or able to use both new and old representations. In a visual task designed to assess the new hand representation, subjects identified visually presented hands as left or right by matching the picture to the representation of their own hands. Functional magnetic resonance imaging showed brain activity in the left posterior frontal cortex (Broca's area) that was unique to the new hand representations of both hands, together with activation in the intraparietal sulcus and prefrontal cortex. The emergence of the new hand representation coincided with the adaptation of perceived location of visible objects in space. These results suggest that the hand representation operates as a visuomotor transformation device that provides an arm-centred frame of reference6 for space perception.
Two male and two female right-handed volunteer psychology students (aged 20–24) served as subjects. They wore left–right reversing spectacles continuously for 39 (two males) or 35 (two females) days. The visual field was about 70° horizontally with 20° binocular overlap, and 60° vertically.
Although visually guided behaviour was disrupted at the beginning of prism wearing, all the subjects could ride a bicycle after one month of adaptation (see Supplementary Information). We examined visually guided hand movements by asking the subjects to reach a visual target presented at various positions for 500 ms. Both error and latency returned to the pre-exposure level within 2 weeks (Fig. 1a), indicating relatively fast adaptation. In contrast, restoration of the perceived location of visible objects was much slower. A visual target was presented on either the left or right side of a display. During the first 2 weeks, the subjects could not correctly report on which side the target was displayed ( Fig. 1b). Although they regained correct localization by day 24, reaction time (RT) indicated a slower adaptation for the left (non-preferred hand) side even on day 35. This time course of perceptual adaptation coincided with the emergence of new body image described below.
We examined the subjects’ body image (activated state of body representation) by asking them to identify pictures of hands viewed from different angles as a left or right hand (Fig. 2i). Usually, subjects solve this task by mentally moving an image of their own hand from its ‘home position’ into a match with the visual stimulus7,8,9,10. We expected that the subjects’ responses would be incorrect at the beginning of prism wearing because the prism reverses the appearance of stimuli, and that correct responses would occur as a new hand representation was established in long-term memory (Fig. 3a, b, c).
Before wearing the prism, responses were almost 100% correct ( Fig. 2a). Early in prism wearing, correct responses essentially disappeared (Fig. 2b, c). However, after 3 weeks of adaptation, correct responses reappeared, suggesting the emergence of a new hand representation. On day 25, there were more correct responses for right-hand stimuli, especially in prototypical orientations (‘home position’8) around 0° or 315° ( Fig. 2d). This result may indicate that the new hand representation is generated earlier for the preferred hand in its most familiar orientation and that the new representation is most easily activated for matching when the stimulus orientation is close to it. At this point, the subjects reported that they could visualize their own hands in two ways: in the old relationship with somatic cues (Fig. 3b), and in the reversed relationship (Fig. 3c), suggesting that the old and new representations coexisted.
Despite the double representations, the subjects could selectively use the new pair if necessary. On day 25, another session of the hand identification task was given with the instruction to “use only your new hand image in all the trials”. The subjects could follow this instruction, producing mostly correct responses (Fig. 2f). Although the RT became much longer, the increase was systematic, indicating that the new representation was acquired earlier for the preferred hand in its prototypical orientation. The steep slope of the RT function would represent awkwardness in mentally moving the novel hand image at this stage (see Fig. 2g for a later stage).
At the final stage of exposure, the RT function was different from that in the pre-test, showing increased RTs for stimuli around 0° ( Fig. 2e). Additional data (Fig. 2g, h) indicate that the increase may be mainly due to conflict between the new and old hand images at these familiar orientations, perhaps because the instruction emphasized attention to both new and old images. The subjects reported that the task would have been much simpler at this stage ( Fig. 2e) if they had been allowed to concentrate on either the new or old images, as they were during the functional magnetic resonance imaging (fMRI) trials (Fig. 2g, h).
To assess the neuronal differences between the old and new hand representations, we scanned each subject's whole brain using fMRI at the final stage of adaptation (days 34–38). We tested the subjects using the hand identification task and explicitly instructed them to use either the new or old hand image only. In each instruction condition, there were also two stimulus identity conditions, a left-hand condition (a left hand was presented in 81% of the trials) and a right-hand condition (a right hand was presented in 81% of the trials).
For each of the four conditions, we analysed the fMRI signals during the test periods against those during control periods. The areas that were active for all of the conditions in common were the parietal association cortex (Brodman's area (BA) 7 or superior parietal lobule), premotor cortex (BA6), visual association cortex (BA18, 19, 37) and cerebellum (Fig. 4a–d). These areas are responsible for visuospatial processing and motor planning11. The involvement of motor planning was indicated by substantial activation in the premotor cortex (BA6)9,10,12, which is not active for purely visual images (as in mental rotation of letters or figures)12,13. Compared with the ‘old’ instruction, the ‘new’ instruction resulted in much more activation in the premotor, parietal and prefrontal cortices (Fig. 4c, d). Activation in the inferior part of the posterior frontal cortex (BA44), intraparietal sulcus (IPS; BA7/39) and prefrontal cortex (BA47, 46, 9) were unique to the ‘new’ instruction (Fig. 4c, d). The activity in the inferior posterior frontal cortex was limited to the left hemisphere regardless of whether the imagined hand was the left or right hand. From the sulcal pattern, the region of activity was likely to be in Broca's area (BA44; Fig. 4e–h). These results indicate that accessing the new hand image is mediated by parieto-frontal circuits including the IPS, Broca's area and the prefrontal cortex14,15.
Presumably the hand representation (memory storage of visuomotor mapping, in our definition) per se cannot be localized in certain regions in the brain. Rather, we may only observe the representation as widespread connections when it is active. Supporting this view, there are anatomically identified parieto-frontal circuits in monkey in which BA7 (around the IPS) projects to ventral premotor cortex (PMv) and prefrontal cortex (PF), and PF projects to the PMv14,15. The PF often subserves associative learning16,17 or working memory18,19, and BA7 and PMv are predominantly engaged in visuomotor transformation20,21,22,23,24. The observed activity in the PF (here, BA47, 46, 9) would reflect greater resource allocation for recalling the new hand image, presumably because the visuomotor association in the new mapping is weaker than that in the old. The greater PF activation for the left-hand image than for the right (Fig. 4c, d) is consistent with the behavioural data, indicating slower acquisition of the new image for the left hand (Fig. 2d–f).
The visuomotor function of BA7 has been well documented in both human20,21 and monkey22,23,24, and this area may code the schema of hand25. In contrast, Broca's area in the human brain, widely known for its language function, is less recognized for its visuomotor contribution. However, there is evidence that the human Broca's area is also involved in preparation for imitative hand movements26,27. The observed activity in Broca's area presumably reflects a visuomotor transformation process that is essential for accessing the new hand representation. Consistent with our results, the monkey PMv (part of which is homologous with human Broca's area, according to some researchers28) has been shown to be crucial for prism adaptation29. Also note that human BA7 is related to prism adaptation21. Both areas were active in our ‘new’ instruction, indicating that the new visuomotor mapping may be coded as a network encompassing many areas14.
In our psychophysical data, the visual localization and hand identification tasks showed a similar time course of adaptation, including slower adaptation for the left (Figs 1b, 2). The new hand representation would serve as a frame of reference upon which normal space perception is regained. According to introspective reports, perceptual harmony between the visible and tactile worlds may be restored only when the subject actively intends to reference the visible world to his or her own body1,2. We infer that the new hand representation contains reversed visuomotor mapping (Fig. 3c), and operates as a visuomotor transformation device that transforms visual images (of objects) into reversed motor output. If the reversed mapping operates on visual input coming through the prism, it would produce normal motor output and would therefore virtually cancel out the reversal of visual input. Although visually guided behaviour measured in the reaching task was restored within 2 weeks (Fig. 1a), more complex behaviour such as riding a bicycle was possible only after the emergence of the new hand representation. This may indicate that the hand representation predicts the relationship between visual and motor codes in various contexts.
Space coding would be often arm-centred6 in the normal situation as well as in adaptation to reversed vision, because we recognize the visual world primarily to allow active interaction between objects and our hands.
Subjects were tested with the reversing spectacles on except for the pre- and post-tests, for which they wore only the frame of the spectacles. All subjects gave informed consent before the experiment.
The subject's head was placed on a chin rest at a viewing distance of 45 cm so that the tested eye (right eye for JA, TU and YH; left eye for CH) was located in front of the centre of a touch display. The other eye was covered. A trial began with a fixation point at the centre of the display, which disappeared when the subject pressed and held a key. After a variable period, a visual target (a circle, 16 mm in diameter) was presented for 500 ms at one of nine locations: the centre and eight radial locations (81 mm from the centre). The subjects were instructed to reach the target as fast and accurately as possible using the preferred (right) hand. The latency (for the hand to depart from the ready position), movement time and the coordinates of the location reached were recorded.
Visual localization task
The subject sat in front of a display with a viewing distance of 105 cm. He/she placed both hands on a switch box that was hidden from view on the subject's knees. A trial began with a fixation point at the centre of the display being displayed for 500 ms. After a 100-ms period with no display, a visual target (a cross, 3 cm × 3 cm) was presented for 200 ms at either 10.5 cm left or right of the centre. The subjects were instructed to press the button that they perceived to be on the same side (left or right) as the visual target, as fast as possible.
Hand identification task (RT experiment)
The subject's head was placed on a chin rest at a viewing distance of 45 cm. Their hands were kept stationary and hidden on their knees under a table. Stimuli consisted of three line drawings of a hand (Fig. 2i) that were presented as either a left or right hand in one of eight orientations in the picture plane (Fig. 2j). When the subject was ready, a series of the total 48 stimuli began in random order. The subjects were instructed to identify each hand as left or right and report the handedness verbally. A stimulus stayed visible until the onset of the subject's voice. During prism wearing, the ordinary instruction asked the subjects to select the more vivid image, in case they had both old and new hand images. In the new-image condition on day 25, they were instructed to exclusively attend to the new hand image.
Echo planar MRI data were acquired with a 1.5-T Siemens Vision system with a standard CP head coil. T2*-weighted functional images were acquired with an in-plane resolution of 2.34 mm × 2.34 mm (repetition time 5.3 s; echo time 66 ms) at each of 16 axial noncontiguous 7-mm-thick slices (with 3.5-mm interslice gaps). In each functional sequence, eight test and eight control scans alternated for a total of 120 scans. In the test periods, the above hand identification task was given, with one trial every 2.6 s (Fig. 4i ). The subjects were asked to report the handedness of each stimulus within 2.6 s by pressing either a head-side or foot-side key on the abdomen with the index fingers of both hands simultaneously. The stimulus disappeared upon response or at the end of the 2.6 s. In the control periods, the hand stimuli were replaced by scrambled hands ( Fig. 4j). The subjects were asked to report whether the majority of the lines were near oblique or horizontal/vertical by key pressing. In the left-hand sequence, 81% (13/16) of the stimuli were left hands, and vice versa in the right-hand sequence. The subjects were tested under both new and old image instructions for both left and right hand sequences on days 34–38. A post-test was conducted five months after removal of the reversing spectacles. The post-test was equivalent to the old-image instruction because the subjects claimed that the new hand image did not make sense at that time.
All fMRI data were processed using the SPM99 software package (http://www.fil.ion.ucl.ac.uk). Standard linear image realignment, linear normalization to the stereotactic anatomical space and spatial smoothing (three-dimensional gaussian kernel, 4.7 mm full-width at half-maximum) were successively performed for each subject. Then, the data from subjects were pooled together and group comparisons were performed using the general linear model. Comparing hand identification with orientation judgement, significant increases were tested with t statistics and displayed as statistical parametric maps. Threshold for significance was set at voxel-level P < 0.05, corrected for multiple comparisons.
We thank M. Kato, T. Hayakawa, T. Murata, H. Tanabe and M. Nakatsuka for technical assistance and discussions.
About this article
Long-lasting aftereffect of a single prism adaptation: shifts in vision and proprioception are independent
Experimental Brain Research (2006)