Letter | Published:

Body image as a visuomotor transformation device revealed in adaptation to reversed vision

Nature volume 407, pages 374377 (21 September 2000) | Download Citation

Subjects

Abstract

People adapt with remarkable flexibility to reversal of the visual field caused by prism spectacles1,2. With sufficient time, this adaptation restores visually guided behaviour and perceptual harmony between the visible and tactile worlds1,2,3. Although it has been suggested that seeing one's own body is crucial for adaptation1,2, the underlying mechanisms are unclear. Here we show that a new representation of visuomotor mapping with respect to the hands emerges in a month during adaptation to reversed vision. The subjects become bi-perceptual3,4,5, or able to use both new and old representations. In a visual task designed to assess the new hand representation, subjects identified visually presented hands as left or right by matching the picture to the representation of their own hands. Functional magnetic resonance imaging showed brain activity in the left posterior frontal cortex (Broca's area) that was unique to the new hand representations of both hands, together with activation in the intraparietal sulcus and prefrontal cortex. The emergence of the new hand representation coincided with the adaptation of perceived location of visible objects in space. These results suggest that the hand representation operates as a visuomotor transformation device that provides an arm-centred frame of reference6 for space perception.

Main

Two male and two female right-handed volunteer psychology students (aged 20–24) served as subjects. They wore left–right reversing spectacles continuously for 39 (two males) or 35 (two females) days. The visual field was about 70° horizontally with 20° binocular overlap, and 60° vertically.

Although visually guided behaviour was disrupted at the beginning of prism wearing, all the subjects could ride a bicycle after one month of adaptation (see Supplementary Information). We examined visually guided hand movements by asking the subjects to reach a visual target presented at various positions for 500 ms. Both error and latency returned to the pre-exposure level within 2 weeks (Fig. 1a), indicating relatively fast adaptation. In contrast, restoration of the perceived location of visible objects was much slower. A visual target was presented on either the left or right side of a display. During the first 2 weeks, the subjects could not correctly report on which side the target was displayed ( Fig. 1b). Although they regained correct localization by day 24, reaction time (RT) indicated a slower adaptation for the left (non-preferred hand) side even on day 35. This time course of perceptual adaptation coincided with the emergence of new body image described below.

Figure 1: Performance in the reaching and visual localization task.
Figure 1

a, Mean latency and reaching error for left, centre and right targets. Reaching error was defined as the horizontal distance between the target and reached position (pixels on a 20-inch display with 640 × 400 resolution). Positive errors indicate reaching to the right of targets, negative errors indicate reaching to the left. One subject was excluded from averaging owing to a recording failure. b, Mean reaction time (RT) and per cent correct localization for the left and right targets. Ps1: post1, within an hour of the removal of the reversing spectacles. Ps2: post2, the day after removal.

We examined the subjects’ body image (activated state of body representation) by asking them to identify pictures of hands viewed from different angles as a left or right hand (Fig. 2i). Usually, subjects solve this task by mentally moving an image of their own hand from its ‘home position’ into a match with the visual stimulus7,8,9,10. We expected that the subjects’ responses would be incorrect at the beginning of prism wearing because the prism reverses the appearance of stimuli, and that correct responses would occur as a new hand representation was established in long-term memory (Fig. 3a, b, c).

Figure 2: The hand identification task.
Figure 2

af, Reaction time (RT) experiment. Mean RT and per cent correct identification are shown as a function of orientation of the stimuli, for the pre-test (a), the ordinary instruction in adaptation period (be), and the ‘new-image’ instruction (f). g,h, Performance during fMRI scans (days 34–38, see Methods). Owing to inability to respond within the 2.6-s time limit, 12% of the trials for the ‘new’ instruction, and 3% for the ‘old’, resulted in missing RT. Analyses of variance (ANOVAs) showed that the ‘new’ instruction tended to yield longer RT (P < 0.075) and fewer correct responses (P < 0.075) than did the ‘old’. i, Examples of stimuli. j, Eight stimulus orientations and two prototypical orientations (0° and 315°) for the right hand.

Figure 3: Schematic description of visuomotor experience for a subject's own right hand, before and while wearing the prisms.
Figure 3

In our definition, the hand representation stores contingency between motor commands and visual images of hand. a, Before wearing prisms. Because normal contingency is stored (parallel lines), the visual image of the right hand is the same as the visible right hand. b, Wearing the prisms. The normal (old) contingency leads to an expected view (visual image) of the right hand that is the reverse of the visible hand for the prism wearer. Thus the subject experiences discrepancy. c, After adaptation, the new contingency (purple lines) is stored. This leads to a new visual image that is identical to the visible one. Thus the subject perceives harmony.

Before wearing the prism, responses were almost 100% correct ( Fig. 2a). Early in prism wearing, correct responses essentially disappeared (Fig. 2b, c). However, after 3 weeks of adaptation, correct responses reappeared, suggesting the emergence of a new hand representation. On day 25, there were more correct responses for right-hand stimuli, especially in prototypical orientations (‘home position’8) around 0° or 315° ( Fig. 2d). This result may indicate that the new hand representation is generated earlier for the preferred hand in its most familiar orientation and that the new representation is most easily activated for matching when the stimulus orientation is close to it. At this point, the subjects reported that they could visualize their own hands in two ways: in the old relationship with somatic cues (Fig. 3b), and in the reversed relationship (Fig. 3c), suggesting that the old and new representations coexisted.

Despite the double representations, the subjects could selectively use the new pair if necessary. On day 25, another session of the hand identification task was given with the instruction to “use only your new hand image in all the trials”. The subjects could follow this instruction, producing mostly correct responses (Fig. 2f). Although the RT became much longer, the increase was systematic, indicating that the new representation was acquired earlier for the preferred hand in its prototypical orientation. The steep slope of the RT function would represent awkwardness in mentally moving the novel hand image at this stage (see Fig. 2g for a later stage).

At the final stage of exposure, the RT function was different from that in the pre-test, showing increased RTs for stimuli around 0° ( Fig. 2e). Additional data (Fig. 2g, h) indicate that the increase may be mainly due to conflict between the new and old hand images at these familiar orientations, perhaps because the instruction emphasized attention to both new and old images. The subjects reported that the task would have been much simpler at this stage ( Fig. 2e) if they had been allowed to concentrate on either the new or old images, as they were during the functional magnetic resonance imaging (fMRI) trials (Fig. 2g, h).

To assess the neuronal differences between the old and new hand representations, we scanned each subject's whole brain using fMRI at the final stage of adaptation (days 34–38). We tested the subjects using the hand identification task and explicitly instructed them to use either the new or old hand image only. In each instruction condition, there were also two stimulus identity conditions, a left-hand condition (a left hand was presented in 81% of the trials) and a right-hand condition (a right hand was presented in 81% of the trials).

For each of the four conditions, we analysed the fMRI signals during the test periods against those during control periods. The areas that were active for all of the conditions in common were the parietal association cortex (Brodman's area (BA) 7 or superior parietal lobule), premotor cortex (BA6), visual association cortex (BA18, 19, 37) and cerebellum (Fig. 4a–d). These areas are responsible for visuospatial processing and motor planning11. The involvement of motor planning was indicated by substantial activation in the premotor cortex (BA6)9,10,12, which is not active for purely visual images (as in mental rotation of letters or figures)12,13. Compared with the ‘old’ instruction, the ‘new’ instruction resulted in much more activation in the premotor, parietal and prefrontal cortices (Fig. 4c, d). Activation in the inferior part of the posterior frontal cortex (BA44), intraparietal sulcus (IPS; BA7/39) and prefrontal cortex (BA47, 46, 9) were unique to the ‘new’ instruction (Fig. 4c, d). The activity in the inferior posterior frontal cortex was limited to the left hemisphere regardless of whether the imagined hand was the left or right hand. From the sulcal pattern, the region of activity was likely to be in Broca's area (BA44; Fig. 4e–h). These results indicate that accessing the new hand image is mediated by parieto-frontal circuits including the IPS, Broca's area and the prefrontal cortex14,15.

Figure 4: Functional MRI scans during mental rotation of a subject's own hand image.
Figure 4

Significantly activated voxels (P < 0.05, corrected) are shown as sagittal projections for each condition (ad). The activity in BA44 is also shown in section displays (eh) with Talairach coordinates for the maxima (see also Supplementary Information) (indicated with red markers in c and d). Example stimuli are shown for the test (i) and control periods (j).

Presumably the hand representation (memory storage of visuomotor mapping, in our definition) per se cannot be localized in certain regions in the brain. Rather, we may only observe the representation as widespread connections when it is active. Supporting this view, there are anatomically identified parieto-frontal circuits in monkey in which BA7 (around the IPS) projects to ventral premotor cortex (PMv) and prefrontal cortex (PF), and PF projects to the PMv14,15. The PF often subserves associative learning16,17 or working memory18,19, and BA7 and PMv are predominantly engaged in visuomotor transformation20,21,22,23,24. The observed activity in the PF (here, BA47, 46, 9) would reflect greater resource allocation for recalling the new hand image, presumably because the visuomotor association in the new mapping is weaker than that in the old. The greater PF activation for the left-hand image than for the right (Fig. 4c, d) is consistent with the behavioural data, indicating slower acquisition of the new image for the left hand (Fig. 2d–f).

The visuomotor function of BA7 has been well documented in both human20,21 and monkey22,23,24, and this area may code the schema of hand25. In contrast, Broca's area in the human brain, widely known for its language function, is less recognized for its visuomotor contribution. However, there is evidence that the human Broca's area is also involved in preparation for imitative hand movements26,27. The observed activity in Broca's area presumably reflects a visuomotor transformation process that is essential for accessing the new hand representation. Consistent with our results, the monkey PMv (part of which is homologous with human Broca's area, according to some researchers28) has been shown to be crucial for prism adaptation29. Also note that human BA7 is related to prism adaptation21. Both areas were active in our ‘new’ instruction, indicating that the new visuomotor mapping may be coded as a network encompassing many areas14.

In our psychophysical data, the visual localization and hand identification tasks showed a similar time course of adaptation, including slower adaptation for the left (Figs 1b, 2). The new hand representation would serve as a frame of reference upon which normal space perception is regained. According to introspective reports, perceptual harmony between the visible and tactile worlds may be restored only when the subject actively intends to reference the visible world to his or her own body1,2. We infer that the new hand representation contains reversed visuomotor mapping (Fig. 3c), and operates as a visuomotor transformation device that transforms visual images (of objects) into reversed motor output. If the reversed mapping operates on visual input coming through the prism, it would produce normal motor output and would therefore virtually cancel out the reversal of visual input. Although visually guided behaviour measured in the reaching task was restored within 2 weeks (Fig. 1a), more complex behaviour such as riding a bicycle was possible only after the emergence of the new hand representation. This may indicate that the hand representation predicts the relationship between visual and motor codes in various contexts.

Space coding would be often arm-centred6 in the normal situation as well as in adaptation to reversed vision, because we recognize the visual world primarily to allow active interaction between objects and our hands.

Methods

Subjects were tested with the reversing spectacles on except for the pre- and post-tests, for which they wore only the frame of the spectacles. All subjects gave informed consent before the experiment.

Reaching task

The subject's head was placed on a chin rest at a viewing distance of 45 cm so that the tested eye (right eye for JA, TU and YH; left eye for CH) was located in front of the centre of a touch display. The other eye was covered. A trial began with a fixation point at the centre of the display, which disappeared when the subject pressed and held a key. After a variable period, a visual target (a circle, 16 mm in diameter) was presented for 500 ms at one of nine locations: the centre and eight radial locations (81 mm from the centre). The subjects were instructed to reach the target as fast and accurately as possible using the preferred (right) hand. The latency (for the hand to depart from the ready position), movement time and the coordinates of the location reached were recorded.

Visual localization task

The subject sat in front of a display with a viewing distance of 105 cm. He/she placed both hands on a switch box that was hidden from view on the subject's knees. A trial began with a fixation point at the centre of the display being displayed for 500 ms. After a 100-ms period with no display, a visual target (a cross, 3 cm × 3 cm) was presented for 200 ms at either 10.5 cm left or right of the centre. The subjects were instructed to press the button that they perceived to be on the same side (left or right) as the visual target, as fast as possible.

Hand identification task (RT experiment)

The subject's head was placed on a chin rest at a viewing distance of 45 cm. Their hands were kept stationary and hidden on their knees under a table. Stimuli consisted of three line drawings of a hand (Fig. 2i) that were presented as either a left or right hand in one of eight orientations in the picture plane (Fig. 2j). When the subject was ready, a series of the total 48 stimuli began in random order. The subjects were instructed to identify each hand as left or right and report the handedness verbally. A stimulus stayed visible until the onset of the subject's voice. During prism wearing, the ordinary instruction asked the subjects to select the more vivid image, in case they had both old and new hand images. In the new-image condition on day 25, they were instructed to exclusively attend to the new hand image.

fMRI measurement

Echo planar MRI data were acquired with a 1.5-T Siemens Vision system with a standard CP head coil. T2*-weighted functional images were acquired with an in-plane resolution of 2.34 mm × 2.34 mm (repetition time 5.3 s; echo time 66 ms) at each of 16 axial noncontiguous 7-mm-thick slices (with 3.5-mm interslice gaps). In each functional sequence, eight test and eight control scans alternated for a total of 120 scans. In the test periods, the above hand identification task was given, with one trial every 2.6 s (Fig. 4i ). The subjects were asked to report the handedness of each stimulus within 2.6 s by pressing either a head-side or foot-side key on the abdomen with the index fingers of both hands simultaneously. The stimulus disappeared upon response or at the end of the 2.6 s. In the control periods, the hand stimuli were replaced by scrambled hands ( Fig. 4j). The subjects were asked to report whether the majority of the lines were near oblique or horizontal/vertical by key pressing. In the left-hand sequence, 81% (13/16) of the stimuli were left hands, and vice versa in the right-hand sequence. The subjects were tested under both new and old image instructions for both left and right hand sequences on days 34–38. A post-test was conducted five months after removal of the reversing spectacles. The post-test was equivalent to the old-image instruction because the subjects claimed that the new hand image did not make sense at that time.

All fMRI data were processed using the SPM99 software package (http://www.fil.ion.ucl.ac.uk). Standard linear image realignment, linear normalization to the stereotactic anatomical space and spatial smoothing (three-dimensional gaussian kernel, 4.7 mm full-width at half-maximum) were successively performed for each subject. Then, the data from subjects were pooled together and group comparisons were performed using the general linear model. Comparing hand identification with orientation judgement, significant increases were tested with t statistics and displayed as statistical parametric maps. Threshold for significance was set at voxel-level P < 0.05, corrected for multiple comparisons.

References

  1. 1.

    The formation and transformation of the perceptual world. Psychol. Issues 3 (monogr. 12) 1–173 (1964).

  2. 2.

    Vision without inversion of the retinal image. Psychol. Rev. 4, 341–360; 463–481 ( 1897).

  3. 3.

    Global plasticity in adult visual cortex following reversal of visual input. Nature 380, 523–526 (1996).

  4. 4.

    et al. Adaptation to reversing prisms activates the ipsilateral visual cortex in humans: an fMRI study. Invest. Ophthalmol. Vis. Sci. 40, S820 (1999).

  5. 5.

    Shifts in binaural localization after prolonged exposure to atypical combination of stimuli. Am. J. Psychol. 68, 526– 548 (1955).

  6. 6.

    , & Coding of visual space by premotor neurons. Science 266, 1054–1057 ( 1994).

  7. 7.

    Kinesthetic aspects of mental representations in the identification of left and right hands. Percept. Psychophys. 32, 89–95 (1982).

  8. 8.

    Mental and physical movements of hands: Kinesthetic information preserved in representational systems. Jpn. Psychol. Res. 25, 95–102 (1983).

  9. 9.

    , , & Neural correlates of mental transformations of the body-in-space. Proc. Natl Acad. Sci. USA 92, 11180–11184 (1995).

  10. 10.

    et al. Use of implicit motor imagery for visual shape discrimination as revealed by PET. Nature 375, 54– 58 (1995).

  11. 11.

    Brain Activation (Wiley-Liss, New York, 1993).

  12. 12.

    , , & Mental rotation of objects versus hands: neural mechanisms revealed by positron emission tomography. Psychophysiology 35, 151–161 (1998).

  13. 13.

    & Functional activation of the human brain during mental rotation. Neuropsychologia 35, 111–1118 (1997).

  14. 14.

    , , & Grasping objects: the cortical mechanisms of visuomotor transformation. Trends Neurosci. 18, 314–320 (1995).

  15. 15.

    & Common cortical subcortical targets of the dorsolateral prefrontal and posterior parietal cortices in the rhesus monkey: Evidence for a distributed neural network subserving spatially guided behavior. J. Neurosci. 8, 4049–4068 (1988).

  16. 16.

    , & Role of prefrontal cortex in a network for arbitrary visuomotor mapping. Exp. Brain Res. 1, 114 –129 (2000).

  17. 17.

    & Rule-dependent neuronal activity in the prefrontal cortex. Exp. Brain Res. 126, 315–335 (1999).

  18. 18.

    et al. Temporal dynamics of brain activation during a working memory task. Nature 386, 604–608 (1997).

  19. 19.

    , , & Transient and sustained activity in a distributed neural system for human working memory. Nature 386, 608– 611 (1997).

  20. 20.

    & Optic ataxia: A specific disruption in visuomotor mechanisms. Brain 111, 643–674 (1988).

  21. 21.

    et al. Role of posterior parietal cortex in the recalibration of visually guided reaching. Nature 383, 618 –621 (1996).

  22. 22.

    , & Coding of intention in the posterior parietal cortex. Nature 386, 167–170 (1997).

  23. 23.

    , & Parietal cortex and movement: I. Movement selection and reaching. Exp. Brain Res. 117, 292– 310 (1997).

  24. 24.

    , , , & Parietal cortex neurons of the monkey related to the visual guidance of hand movement. Exp. Brain Res. 83, 29–36 ( 1990).

  25. 25.

    , & Coding of modified body schema during tool use by macaque postcentral neurons. NeuroReport 7, 2325 –2330 (1996).

  26. 26.

    , , , & The preparation, execution and suppression of copied movements in the human brain. Exp. Brain Res. 120, 386–398 (1998).

  27. 27.

    , , & Localization of grasp representations in humans by positron emission tomography: 2. Observation compared with imagination. Exp. Brain Res. 112, 103–111 (1996).

  28. 28.

    , , & Premotor cortex and the recognition of motor actions. Cogn. Brain Res. 3, 131–141 (1996).

  29. 29.

    & Reacquisition deficits in prism adaptation after muscimol microinjection into the ventral premotor cortex of monkeys. J. Neurophysiol. 81, 1927– 1938 (1999).

Download references

Acknowledgements

We thank M. Kato, T. Hayakawa, T. Murata, H. Tanabe and M. Nakatsuka for technical assistance and discussions.

Author information

Affiliations

  1. *Division of Cognitive Psychology, Future University-Hakodate, Kameda Nakano, Hakodate 041-8655, Japan

    • Kaoru Sekiyama
  2. †Auditory & Visual Informatics Section, Kansai Advanced Research Center, Communications Research Laboratory , Iwaoka, Kobe 651-2492, Japan

    • Satoru Miyauchi
    •  & Toshihide Imaruoka
  3. ‡Graduate School of Medicine, Osaka University, Yamadaoka, Suita 565-0781, Japan

    • Toshihide Imaruoka
  4. §School of Health, International University of Health and Welfare, Ohtawara 324-8501, Japan

    • Hiroyuki Egusa
  5. Department of Psychology, Osaka City University, Sugimoto, Osaka 558-8585, Japan

    • Takara Tashiro

Authors

  1. Search for Kaoru Sekiyama in:

  2. Search for Satoru Miyauchi in:

  3. Search for Toshihide Imaruoka in:

  4. Search for Hiroyuki Egusa in:

  5. Search for Takara Tashiro in:

Corresponding author

Correspondence to Kaoru Sekiyama.

Supplementary information

PDF files

  1. 1.

    Figure 1

  2. 2.

    Figure 2

Excel files

  1. 1.

    Table 1

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/35030096

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.