Humans perform goal-directed actions such as reaching for a light switch or grasping a coffee mug thousands of times a day. Behind the scenes of these seemingly simple actions, the brain performs sophisticated calculations to locate the target object of the action and correctly guide the hand towards it. In this Review, we discuss how the brain establishes spatial representations used for visually guided actions. In addition to reviewing simple tasks and paradigms, we discuss spatial coding in complex and naturalistic environments. We highlight the importance of high-level cognitive factors, such as memory, task constraints, and object semantics, which influence the use of spatial representations for action. To move the field forward, we suggest that future research should integrate across different scales of action spaces from small-scale finger movements to large-scale navigation. Doing so would enable the identification of general mechanisms that underlie spatial coding across different actions and spaces.
This is a preview of subscription content, access via your institution
Subscribe to Nature+
Get immediate online access to Nature and 55 other Nature journal
Subscribe to Journal
Get full journal access for 1 year
only $4.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Colby, C. L. Action-oriented spatial reference frames in cortex. Neuron 20, 15–24 (1998).
Klatzky, R. L. in Spatial Cognition: An Interdisciplinary Approach to Representing and Processing Spatial Knowledge (eds Freska, C., Habel, C. & Wender K. F.) 1–17 (Springer, 1998).
Crawford, J. D., Henriques, D. Y. P. & Medendorp, W. P. Three-dimensional transformations for goal-directed action. Annu. Rev. Neurosci. 34, 309–331 (2011).
McGuire, L. M. M. & Sabes, P. N. Sensory transformations and the use of multiple reference frames for reach planning. Nat. Neurosci. 12, 1056–1061 (2009).
O’Regan, J. K. & Noë, A. A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24, 939–973 (2001).
Hommel, B., Müsseler, J., Aschersleben, G. & Prinz, W. The Theory of Event Coding (TEC): a framework for perception and action planning. Behav. Brain Sci. 24, 849–878 (2001).
Rizzolatti, G. & Craighero, L. The mirror-neuron system. Annu. Rev. Neurosci. 27, 169–192 (2004).
Flanders, M., Tillery, S. I. H. & Soechting, J. F. Early stages in a sensorimotor transformation. Behav. Brain Sci. 15, 309–320 (1992).
Blohm, G. & Crawford, J. D. Computations for geometrically accurate visually guided reaching in 3-D space. J. Vis. 7, 4 (2007).
Crawford, J. D. & Guitton, D. Visual–motor transformations required for accurate and kinematically correct saccades. J. Neurophysiol. 78, 1447–1467 (1997).
Henriques, D. Y. P., Klier, E. M., Smith, M. A., Lowy, D. & Crawford, J. D. Gaze-centered remapping of remembered visual space in an open-loop pointing task. J. Neurosci. 18, 1583–1594 (1998).
Batista, A. P., Buneo, C. A., Snyder, L. H. & Andersen, R. A. Reach plans in eye-centered coordinates. Science 285, 257–260 (1999).
Voudouris, D., Smeets, J. B. J., Fiehler, K. & Brenner, E. Gaze when reaching to grasp a glass. J. Vis. 18, 16 (2018).
van Beers, R. J., van Mierlo, C. M., Smeets, J. B. J. & Brenner, E. Reweighting visual cues by touch. J. Vis. 11, 20 (2011).
Camponogara, I. & Volcic, R. Integration of haptics and vision in human multisensory grasping. Cortex. 135, 173–185 (2021).
Cuijpers, R. H., Brenner, E. & Smeets, J. B. J. Consistent haptic feedback is required but it is not enough for natural reaching to virtual cylinders. Hum. Mov. Sci. 27, 857–872 (2008).
Medendorp, W. P. Spatial constancy mechanisms in motor control. Phil. Trans. R. Soc. Lond. B 366, 476–491 (2011).
Blohm, G. et al. Neuromagnetic signatures of the spatiotemporal transformation for manual pointing. Neuroimage 197, 306–319 (2019).
Medendorp, W. P., Beurze, S. M., van Pelt, S. & van der Werf, J. Behavioral and cortical mechanisms for spatial coding and action planning. Cortex 44, 587–597 (2008).
Culham, J. C. & Valyear, K. F. Human parietal cortex in action. Curr. Opin. Neurobiol. 16, 205–212 (2006).
Perenin, M. T. & Vighetto, A. Optic ataxia: a specific disruption in visuomotor mechanisms. I. Different aspects of the deficit in reaching for objects. Brain 111, 643–674 (1988).
Hallett, P. E. Primary and secondary saccades to goals defined by instructions. Vis. Res. 18, 1279–1296 (1978).
Gail, A. & Andersen, R. A. Neural dynamics in monkey parietal reach region reflect context-specific sensorimotor transformations. J. Neurosci. 26, 9376–9384 (2006).
DeSouza, J. F. X. et al. Eye position signal modulates a human parietal pointing region during memory-guided movements. J. Neurosci. 20, 5835–5840 (2000).
Medendorp, W. P., Goltz, H. C., Crawford, J. D. & Vilis, T. Integration of target and effector information in human posterior parietal cortex for the planning of action. J. Neurophysiol. 93, 954–962 (2005).
Gail, A., Klaes, C. & Westendorff, S. Implementation of spatial transformation rules for goal-directed reaching via gain modulation in monkey parietal and premotor cortex. J. Neurosci. 29, 9490–9499 (2009).
Gertz, H., Lingnau, A. & Fiehler, K. Decoding movement goals from the fronto-parietal reach network. Front. Hum. Neurosci. 11, 84 (2017).
Westendorff, S., Klaes, C. & Gail, A. The cortical timeline for deciding on reach motor goals. J. Neurosci. 30, 5426–5436 (2010).
Fernandez-Ruiz, J., Goltz, H. C., DeSouza, J. F. X., Vilis, T. & Crawford, J. D. Human parietal “reach region” primarily encodes intrinsic visual direction, not extrinsic movement direction, in a visual motor dissociation task. Cereb. Cortex 17, 2283–2292 (2007).
Dum, R. P. & Strick, P. L. Frontal lobe inputs to the digit representations of the motor areas on the lateral surface of the hemisphere. J. Neurosci. 25, 1375–1386 (2005).
He, S. Q., Dum, R. P. & Strick, P. L. Topographic organization of corticospinal projections from the frontal lobe: motor areas on the lateral surface of the hemisphere. J. Neurosci. 13, 952–980 (1993).
Connolly, J. D., Goodale, M. A., DeSouza, J. F., Menon, R. S. & Vilis, T. A comparison of frontoparietal fMRI activation during anti-saccades and anti-pointing. J. Neurophysiol. 84, 1645–1655 (2000).
Gertz, H. & Fiehler, K. Human posterior parietal cortex encodes the movement goal in a pro-/anti-reach task. J. Neurophysiol. 114, 170–183 (2015).
Beurze, S. M., Lange, F. P., de, Toni, I. & Medendorp, W. P. Integration of target and effector information in the human brain during reach planning. J. Neurophysiol. 97, 188–199 (2007).
Chapman, C. S. et al. Reaching for the unknown: multiple target encoding and real-time decision-making in a rapid reach task. Cognition 116, 168–176 (2010).
Stewart, B. M., Baugh, L. A., Gallivan, J. P. & Flanagan, J. R. Simultaneous encoding of the direction and orientation of potential targets during reach planning: evidence of multiple competing reach plans. J. Neurophysiol. 110, 807–816 (2013).
Hesse, C., Kangur, K. & Hunt, A. R. Decision making in slow and rapid reaching: sacrificing success to minimize effort. Cognition 205, 104426 (2020).
Onagawa, R. & Kudo, K. Sensorimotor strategy selection under time constraints in the presence of two motor targets with different values. Sci. Rep. 11, 22207 (2021).
Stewart, B. M., Gallivan, J. P., Baugh, L. A. & Flanagan, J. R. Motor, not visual, encoding of potential reach targets. Curr. Biol. 24, R953–R954 (2014).
Gallivan, J. P., Stewart, B. M., Baugh, L. A., Wolpert, D. M. & Flanagan, J. R. Rapid automatic motor encoding of competing reach options. Cell Rep. 18, 1619–1626 (2017).
Gallivan, J. P., Logan, L., Wolpert, D. M. & Flanagan, J. R. Parallel specification of competing sensorimotor control policies for alternative action options. Nat. Neurosci. 19, 320–326 (2016).
Praamstra, P., Kourtis, D. & Nazarpour, K. Simultaneous preparation of multiple potential movements: opposing effects of spatial proximity mediated by premotor and parietal cortex. J. Neurophysiol. 102, 2084–2095 (2009).
Cisek, P. Cortical mechanisms of action selection: the affordance competition hypothesis. Phil. Trans. R. Soc. Lond. B 362, 1585–1599 (2007).
Cisek, P. & Kalaska, J. F. Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45, 801–814 (2005).
Gallivan, J. P., Barton, K. S., Chapman, C. S., Wolpert, D. M. & Flanagan, J. R. Action plan co-optimization reveals the parallel encoding of competing reach movements. Nat. Commun. 6, 7428 (2015).
Alhussein, L. & Smith, M. A. Motor planning under uncertainty. eLife 10, e67019 (2021).
Battaglia-Mayer, A., Caminiti, R., Lacquaniti, F. & Zago, M. Multiple levels of representation of reaching in the parieto-frontal network. Cereb. Cortex 13, 1009–1022 (2003).
Soechting, J. F. & Flanders, M. Moving in three-dimensional space: frames of reference, vectors, and coordinate systems. Annu. Rev. Neurosci. 15, 167–191 (1992).
Thompson, A. A. & Henriques, D. Y. P. The coding and updating of visuospatial memory for goal-directed reaching and pointing. Vis. Res. 51, 819–826 (2011).
Bock, O. Contribution of retinal versus extraretinal signals towards visual localization in goal-directed movements. Exp. Brain Res. 64, 476–482 (1986).
Selen, L. P. J. & Medendorp, W. P. Saccadic updating of object orientation for grasping movements. Vis. Res. 51, 898–907 (2011).
Medendorp, W. P. & Crawford, J. D. Visuospatial updating of reaching targets in near and far space. Neuroreport 13, 633–636 (2002).
Mueller, S. & Fiehler, K. Effector movement triggers gaze-dependent spatial coding of tactile and proprioceptive-tactile reach targets. Neuropsychologia 62, 184–193 (2014).
Lewald, J. & Ehrenstein, W. H. The effect of eye position on auditory lateralization. Exp. Brain Res. 108, 473–485 (1996).
Pouget, A., Deneve, S. & Duhamel, J.-R. A computational perspective on the neural basis of multisensory spatial representations. Nat. Rev. Neurosci. 3, 741–747 (2002).
Medendorp, W. P., Goltz, H. C., Vilis, T. & Crawford, J. D. Gaze-centered updating of visual space in human parietal cortex. J. Neurosci. 23, 6209–6214 (2003).
Leoné, F. T. M., Monaco, S., Henriques, D. Y. P., Toni, I. & Medendorp, W. P. Flexible reference frames for grasp planning in human parietofrontal cortex. eNeuro 2, ENEURO.0008-15.2015 (2015).
Dijkerman, H. C. et al. Reaching errors in optic ataxia are linked to eye position rather than head or body position. Neuropsychologia 44, 2766–2773 (2006).
Khan, A. Z., Pisella, L., Rossetti, Y., Vighetto, A. & Crawford, J. D. Impairment of gaze-centered updating of reach targets in bilateral parietal-occipital damaged patients. Cereb. Cortex 15, 1547–1560 (2005).
Ambrosini, E. et al. Behavioral investigation on the frames of reference involved in visuomotor transformations during peripheral arm reaching. PLoS One 7, e51856 (2012).
Bernier, P.-M. & Grafton, S. T. Human posterior parietal cortex flexibly determines reference frames for reaching based on sensory context. Neuron 68, 776–788 (2010).
Mullette-Gillman, O. A., Cohen, Y. E. & Groh, J. M. Eye-centered, head-centered, and complex coding of visual and auditory targets in the intraparietal sulcus. J. Neurophysiol. 94, 2331–2352 (2005).
Mullette-Gillman, O. A., Cohen, Y. E. & Groh, J. M. Motor-related signals in the intraparietal cortex encode locations in a hybrid, rather than eye-centered reference frame. Cereb. Cortex 19, 1761–1775 (2009).
Obhi, S. S. & Goodale, M. A. The effects of landmarks on the performance of delayed and real-time pointing movements. Exp. Brain Res. 167, 335–344 (2005).
Krigolson, O. & Heath, M. Background visual cues and memory-guided reaching. Hum. Mov. Sci. 23, 861–877 (2004).
Krigolson, O., Clark, N., Heath, M. & Binsted, G. The proximity of visual landmarks impacts reaching performance. Spat. Vis. 20, 317–336 (2007).
Byrne, P. A. & Crawford, J. D. Cue reliability and a landmark stability heuristic determine relative weighting between egocentric and allocentric visual information in memory-guided reach. J. Neurophysiol. 103, 3054–3069 (2010).
Taghizadeh, B. & Gail, A. Spatial task context makes short-latency reaches prone to induced Roelofs illusion. Front. Hum. Neurosci. 8, 673 (2014).
Schenk, T. An allocentric rather than perceptual deficit in patient D.F. Nat. Neurosci. 9, 1369–1370 (2006).
Chen, Y. et al. Allocentric versus egocentric representation of remembered reach targets in human cortex. J. Neurosci. 34, 12515–12526 (2014).
Chen, Y., Monaco, S. & Crawford, J. D. Neural substrates for allocentric-to-egocentric conversion of remembered reach targets in humans. Eur. J. Neurosci. 47, 901–917 (2018).
Taghizadeh, B., Fortmann, O. & Gail, A. Position- and scale-invariant object-centered spatial selectivity in monkey frontoparietal cortex dynamically adapts to task demand. Preprint at bioRxiv https://doi.org/10.1101/2022.01.26.477941 (2022).
Carrozzo, M., Stratta, F., McIntyre, J. & Lacquaniti, F. Cognitive allocentric representations of visual space shape pointing errors. Exp. Brain Res. 147, 426–436 (2002).
Neely, K. A., Tessmer, A., Binsted, G. & Heath, M. Goal-directed reaching: movement strategies influence the weighting of allocentric and egocentric visual cues. Exp. Brain Res. 186, 375–384 (2008).
Thompson, A. A. & Henriques, D. Y. P. Locations of serial reach targets are coded in multiple reference frames. Vis. Res. 50, 2651–2660 (2010).
Ernst, M. O. & Banks, M. S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature. 415, 429–433 (2002).
Knill, D. C. Robust cue integration: a Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant. J. Vis. 7, 5 (2007).
Körding, K. P. & Wolpert, D. M. Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10, 319–326 (2006).
Vaziri, S., Diedrichsen, J. & Shadmehr, R. Why does the brain predict sensory consequences of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback. J. Neurosci. 26, 4188–4197 (2006).
Landy, M. S., Maloney, L. T., Johnston, E. B. & Young, M. Measurement and modeling of depth cue combination: in defense of weak fusion. Vis. Res. 35, 389–412 (1995).
Knill, D. C. & Richards, W. Perception As Bayesian Inference (Cambridge Univ. Press, 1996).
Tagliabue, M. & McIntyre, J. A modular theory of multisensory integration for motor control. Front. Comput. Neurosci. 8, 1 (2014).
Karimpur, H., Kurz, J. & Fiehler, K. The role of perception and action on the use of allocentric information in a large-scale virtual environment. Exp. Brain Res. 238, 1813–1826 (2020).
Klinghammer, M., Blohm, G. & Fiehler, K. Scene configuration and object reliability affect the use of allocentric information for memory-guided reaching. Front. Neurosci. 11, 204 (2017).
Camors, D., Jouffrais, C., Cottereau, B. R. & Durand, J. B. Allocentric coding: spatial range and combination rules. Vis. Res. 109, 87–98 (2015).
Körding, K. P. et al. Causal inference in multisensory perception. PLoS One 2, e943 (2007).
Sato, Y., Toyoizumi, T. & Aihara, K. Bayesian inference explains perception of unity and ventriloquism aftereffect: identification of common sources of audiovisual stimuli. Neural Comput. 19, 3335–3355 (2007).
Diedrichsen, J., Werner, S., Schmidt, T. & Trommershäuser, J. Immediate spatial distortions of pointing movements induced by visual landmarks. Percept. Psychophys. 66, 89–103 (2004).
Neggers, S. F. W., Schölvinck, M. L., van der Lubbe, R. H. J. & Postma, A. Quantifying the interactions between allo- and egocentric representations of space. Acta Psychol. 118, 25–45 (2005).
Ruotolo, F., van der Ham, I. J. M., Iachini, T. & Postma, A. The relationship between allocentric and egocentric frames of reference and categorical and coordinate spatial information processing. Q. J. Exp. Psychol. 64, 1138–1156 (2011).
Goodale, M. A. & Milner, A. Separate visual pathways for perception and action. Trends Neurosci. 15, 20–25 (1992).
Heath, M. & Westwood, D. A. Can a visual representation support the online control of memory-dependent reaching? Evident from a variable spatial mapping paradigm. Mot. Control. 7, 346–361 (2003).
Westwood, D. A., Heath, M. & Roy, E. A. No evidence for accurate visuomotor memory: systematic and variable error in memory-guided reaching. J. Mot. Behav. 35, 127–133 (2003).
Hesse, C. & Franz, V. H. Memory mechanisms in grasping. Neuropsychologia 47, 1532–1545 (2009).
Hesse, C. & Franz, V. H. Grasping remembered objects: exponential decay of the visual memory. Vis. Res. 50, 2642–2650 (2010).
Franz, V. H., Hesse, C. & Kollath, S. Visual illusions, delayed grasping, and memory: no shift from dorsal to ventral control. Neuropsychologia 47, 1518–1531 (2009).
Fiehler, K. et al. Working memory maintenance of grasp-target information in the human posterior parietal cortex. Neuroimage 54, 2401–2411 (2011).
Freud, E., Plaut, D. C. & Behrmann, M. ‘What’ is happening in the dorsal visual pathway. Trends Cogn. Sci. 20, 773–784 (2016).
Himmelbach, M. et al. Brain activation during immediate and delayed reaching in optic ataxia. Neuropsychologia 47, 1508–1517 (2009).
Goodale, M. Frames of reference for perception and action in the human visual system. Neurosci. Biobehav. Rev. 22, 161–172 (1998).
Westwood, D. A. & Goodale, M. A. Perceptual illusion and the real-time control of action. Spat. Vis. 16, 243–254 (2003).
Schenk, T. & McIntosh, R. D. Do we have independent visual streams for perception and action? Cogn. Neurosci. 1, 52–62 (2010).
Schenk, T. & Hesse, C. Do we have distinct systems for immediate and delayed actions? A selective review on the role of visual memory in action. Cortex 98, 228–248 (2018).
Schütz, I., Henriques, D. Y. P. & Fiehler, K. Gaze-centered spatial updating in delayed reaching even in the presence of landmarks. Vis. Res. 87, 46–52 (2013).
Schütz, I., Henriques, D. Y. P. & Fiehler, K. No effect of delay on the spatial representation of serial reach targets. Exp. Brain Res. 233, 1225–1235 (2015).
Bridgemen, B., Kirch, M. & Sperling, A. Segregation of cognitive and motor aspects of visual function using induced motion. Percept. Psychophys. 29, 336–342 (1981).
Brenner, E. & Smeets, J. B. Fast responses of the human hand to changes in target position. J. Mot. Behav. 29, 297–310 (1997).
Grave, D. D. J., de, Brenner, E. & Smeets, J. B. J. Illusions as a tool to study the coding of pointing movements. Exp. Brain Res. 155, 56–62 (2004).
Gomi, H. Implicit online corrections of reaching movements. Curr. Opin. Neurobiol. 18, 558–564 (2008).
Lu, Z. & Fiehler, K. Spatial updating of allocentric landmark information in real-time and memory-guided reaching. Cortex 125, 203–214 (2020).
Bridgeman, B., Peery, S. & Anand, S. Interaction of cognitive and sensorimotor maps of visual space. Percept. Psychophys. 59, 456–469 (1997).
Chen, Y., Byrne, P. & Crawford, J. D. Time course of allocentric decay, egocentric decay, and allocentric-to-egocentric conversion in memory-guided reach. Neuropsychologia 49, 49–60 (2011).
Hay, L. & Redon, C. Response delay and spatial representation in pointing movements. Neurosci. Lett. 408, 194–198 (2006).
Sheth, B. R. & Shimojo, S. Extrinsic cues suppress the encoding of intrinsic cues. J. Cogn. Neurosci. 16, 339–350 (2004).
Crowe, E. M. et al. Further evidence that people rely on egocentric information to guide a cursor to a visible target. Perception 50, 904–907 (2021).
Crowe, E. M., Bossard, M. & Brenner, E. Can ongoing movements be guided by allocentric visual information when the target is visible? J. Vis. 21, 6 (2021).
Land, M. F. & Hayhoe, M. In what ways do eye movements contribute to everyday activities? Vis. Res. 41, 3559–3565 (2001).
Rothkopf, C. A., Ballard, D. H. & Hayhoe, M. M. Task and context determine where you look. J. Vis. 7, 16 (2007).
Broadbent, D. E. Perception And Communication (Pergamon Press, 1958).
Bisley, J. W. & Goldberg, M. E. Attention, intention, and priority in the parietal lobe. Annu. Rev. Neurosci. 33, 1–21 (2010).
Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995).
Fecteau, J. H. & Munoz, D. P. Salience, relevance, and firing: a priority map for target selection. Trends Cogn. Sci. 10, 382–390 (2006).
Carrasco, M. Visual attention: the past 25 years. Vis. Res. 51, 1484–1525 (2011).
Moehler, T. & Fiehler, K. Effects of spatial congruency on saccade and visual discrimination performance in a dual-task paradigm. Vis. Res. 105, 100–111 (2014).
Moehler, T. & Fiehler, K. The influence of spatial congruency and movement preparation time on saccade curvature in simultaneous and sequential dual-tasks. Vis. Res. 116, 25–35 (2015).
Behrmann, M. & Tipper, S. P. Attention accesses multiple reference frames: evidence from visual neglect. J. Exp. Psychol. Hum. Percept. Perform. 25, 83–101 (1999).
Chun, M. M., Golomb, J. D. & Turk-Browne, N. B. A taxonomy of external and internal attention. Annu. Rev. Psychol. 62, 73–101 (2011).
Abrams, R. A. & Dobkin, R. S. Inhibition of return: effects of attentional cuing on eye movement latencies. J. Exp. Psychol. Hum. Percept. Perform. 20, 467–477 (1994).
Tipper, S. P., Weaver, B., Jerreat, L. M. & Burak, A. L. Object-based and environment-based inhibition of return of visual attention. J. Exp. Psychol. Hum. Percept. Perform. 20, 478–499 (1994).
Klinghammer, M., Blohm, G. & Fiehler, K. Contextual factors determine the use of allocentric information for reaching in a naturalistic scene. J. Vis. 15, 24 (2015).
Posner, M. I. Orienting of attention. Q. J. Exp. Psychol. 32, 3–25 (1980).
Ballard, D. H. & Hayhoe, M. M. Modelling the role of task in the control of gaze. Vis. Cogn. 17, 1185–1204 (2009).
Mills, M., Hollingworth, A., van der Stigchel, S., Hoffman, L. & Dodd, M. D. Examining the influence of task set on eye movements and fixations. J. Vis. 11, 17 (2011).
Golomb, J. D., Pulido, V. Z., Albrecht, A. R., Chun, M. M. & Mazer, J. A. Robustness of the retinotopic attentional trace after eye movements. J. Vis. 10, 19 (2010).
Jonikaitis, D. & Moore, T. The interdependence of attention, working memory and gaze control: behavior and neural circuitry. Curr. Opin. Psychol. 29, 126–134 (2019).
Maxcey-Richard, A. M. & Hollingworth, A. The strategic retention of task-relevant objects in visual working memory. J. Exp. Psychol. Learn. Mem. Cogn. 39, 760–772 (2013).
Schneider, W. X., Einhäuser, W. & Horstmann, G. Attentional selection in visual perception, memory and action: a quest for cross-domain integration. Phil. Trans. R. Soc. Lond. B 368, 20130053 (2013).
Lu, Z., Klinghammer, M. & Fiehler, K. The role of gaze and prior knowledge on allocentric coding of reach targets. J. Vis. 18, 22 (2018).
Jiang, Y. V., Swallow, K. M. & Sun, L. Egocentric coding of space for incidentally learned attention: effects of scene context and task instructions. J. Exp. Psychol. Learn. Mem. Cogn. 40, 233–250 (2014).
Henderson, J. M. Human gaze control during real-world scene perception. Trends Cogn. Sci. 7, 498–504 (2003).
Henderson, J. M. Gaze control as prediction. Trends Cogn. Sci. 21, 15–23 (2017).
Võ, M. L.-H. & Wolfe, J. M. Differential electrophysiological signatures of semantic and syntactic scene processing. Psychol. Sci. 24, 1816–1823 (2013).
Hayes, T. R. & Henderson, J. M. Scene semantics involuntarily guide attention during visual search. Psychon. Bull. Rev. 26, 1683–1689 (2019).
Henderson, J. M. & Hayes, T. R. Meaning-based guidance of attention in scenes as revealed by meaning maps. Nat. Hum. Behav. 1, 743–747 (2017).
Oliva, A. & Torralba, A. Chapter 2 Building the gist of a scene: the role of global image features in recognition. Prog. Brain Res. 155, 23–36 (2006).
Cornelissen, T. H. W. & Võ, M. L.-H. Stuck on semantics: processing of irrelevant object–scene inconsistencies modulates ongoing gaze behavior. Atten. Percept. Psychophys. 79, 154–168 (2017).
Malcolm, G. L., Rattinger, M. & Shomstein, S. Intrusive effects of semantic information on visual selective attention. Atten. Percept. Psychophys. 78, 2066–2078 (2016).
Peacock, C. E., Hayes, T. R. & Henderson, J. M. Meaning guides attention during scene viewing, even when it is irrelevant. Atten. Percept. Psychophys. 81, 20–34 (2019).
Glover, S. & Dixon, P. Semantics affect the planning but not control of grasping. Exp. Brain Res. 146, 383–387 (2002).
Naylor, C. E., Power, T. J. & Buckingham, G. Examining whether semantic cues can affect felt heaviness when lifting novel objects. J. Cogn. 3, 3 (2020).
Yantis, S. Multielement visual tracking: attention and perceptual organization. Cogn. Psychol. 24, 295–340 (1992).
Hock, H. S., Gordon, G. P. & Whitehurst, R. Contextual relations: the influence of familiarity, physical plausibility, and belongingness. Percept. Psychophys. 16, 4–8 (1974).
Karimpur, H., Morgenstern, Y. & Fiehler, K. Facilitation of allocentric coding by virtue of object-semantics. Sci. Rep. 9, 6263 (2019).
Kriegeskorte, N. & Mur, M. Inverse MDS: inferring dissimilarity structure from multiple item arrangements. Front. Psychol. 3, 245 (2012).
Goldstein, E. B. Spatial layout, orientation relative to the observer, and perceived projection in pictures viewed at an angle. J. Exp. Psychol. Hum. Percept. Perform. 13, 256–266 (1987).
Koenderink, J. J. & van Doorn, A. in Looking Into Pictures: An Interdisciplinary Approach To Pictorial Space (eds Hecht, H., Schwartz, R. & Atherton, M.) 239–299 (MIT Press, 2003).
Koenderink, J. & van Doorn, A. The structure of visual spaces. J. Math. Imaging Vis. 31, 171–187 (2008).
Vishwanath, D., Girshick, A. R. & Banks, M. S. Why pictures look right when viewed from the wrong place. Nat. Neurosci. 8, 1401–1410 (2005).
Heidegger, M. Being And Time: A Translation Of Sein Und Zeit (Suny Press, 1996).
Wollheim, R. Wollheim on pictorial representation. J. Aesthet. Art. Crit. 56, 217–226 (1998).
Troje, N. F. Reality check. Perception. 48, 1033–1038 (2019).
Karimpur, H., Eftekharifar, S., Troje, N. F. & Fiehler, K. Spatial coding for memory-guided reaching in visual and pictorial spaces. J. Vis. 20, 1 (2020).
Mountcastle, V. B., Lynch, J. C., Georgopoulos, A., Sakata, H. & Acuna, C. Posterior parietal association cortex of the monkey: command functions for operations within extrapersonal space. J. Neurophysiol. 38, 871–908 (1975).
Rizzolatti, G., Scandolara, C., Matelli, M. & Gentilucci, M. Afferent properties of periarcuate neurons in macaque monkeys. I. Somatosensory responses. Behav. Brain Res. 2, 125–146 (1981).
Rizzolatti, G., Fadiga, L., Fogassi, L. & Gallese, V. The space around us. Science 277, 190–191 (1997).
Serino, A. et al. Body part-centered and full body-centered peripersonal space representations. Sci. Rep. 5, 18603 (2015).
Iriki, A., Tanaka, M. & Iwamura, Y. Coding of modified body schema during tool use by macaque postcentral neurones. Neuroreport 7, 2325–2330 (1996).
Farnè, A., Serino, A. & Làdavas, E. Dynamic size-change of peri-hand space following tool-use: determinants and spatial characteristics revealed through cross-modal extinction. Cortex 43, 436–443 (2007).
Longo, M. R. & Lourenco, S. F. On the nature of near space: effects of tool use and the transition to far space. Neuropsychologia 44, 977–981 (2006).
Mine, D. & Yokosawa, K. Disconnected hand avatar can be integrated into the peripersonal space. Exp. Brain Res. 239, 237–244 (2021).
Bufacchi, R. J. & Iannetti, G. D. An action field theory of peripersonal space. Trends Cogn. Sci. 22, 1076–1090 (2018).
Noel, J.-P. et al. Full body action remapping of peripersonal space: the case of walking. Neuropsychologia 70, 375–384 (2015).
Montello, D. R. in Spatial Information Theory: A Theoretical Basis For GIS (eds Frank, A. U. & Campari, I.) 312–321 (Proceedings of COSIT’93, Springer, 1993).
Fiehler, K., Wolf, C., Klinghammer, M. & Blohm, G. Integration of egocentric and allocentric information during memory-guided reaching to images of a natural environment. Front. Hum. Neurosci. 8, 636 (2014).
Klinghammer, M., Schütz, I., Blohm, G. & Fiehler, K. Allocentric information is used for memory-guided reaching in depth: a virtual reality study. Vis. Res. 129, 13–24 (2016).
Sadeh, M., Sajad, A., Wang, H., Yan, X. & Crawford, J. D. The influence of a memory delay on spatial coding in the superior colliculus: is visual always visual and motor always motor? Front. Neural Circuits 12, 74 (2018).
Sadeh, M., Sajad, A., Wang, H., Yan, X. & Crawford, J. D. Spatial transformations between superior colliculus visual and motor response fields during head-unrestrained gaze shifts. Eur. J. Neurosci. 42, 2934–2951 (2015).
Sajad, A. et al. Visual–motor transformations within frontal eye fields during head-unrestrained gaze shifts in the monkey. Cereb. Cortex 25, 3932–3952 (2015).
Berger, M., Agha, N. S. & Gail, A. Wireless recording from unrestrained monkeys reveals motor goal encoding beyond immediate reach in frontoparietal cortex. eLife 9, e51322 (2020).
Draschkow, D., Nobre, A. C. & van Ede, F. Multiple spatial frames for immersive working memory. Nat. Hum. Behav. 6, 536–544 (2022).
Meilinger, T., Riecke, B. E. & Bülthoff, H. H. Local and global reference frames for environmental spaces. Q. J. Exp. Psychol. 67, 542–569 (2014).
Meilinger, T., Strickrodt, M. & Bülthoff, H. H. Qualitative differences in memory for vista and environmental spaces are caused by opaque borders, not movement or successive presentation. Cognition 155, 77–95 (2016).
Treisman, A. M. & Gelade, G. A feature-integration theory of attention. Cogn. Psychol. 12, 97–136 (1980).
van Gomple, R. P. G. Eye Movements (Elsevier, 2007).
Wolfe, J. M., Võ, M. L.-H., Evans, K. K. & Greene, M. R. Visual search in scenes involves selective and nonselective pathways. Trends Cogn. Sci. 15, 77–84 (2011).
Engel, A. K., Maye, A., Kurthen, M. & König, P. Where’s the action? The pragmatic turn in cognitive science. Trends Cogn. Sci. 17, 202–209 (2013).
Adam, J. J., Bovend’Eerdt, T. J. H., Schuhmann, T. & Sack, A. T. Allocentric coding in ventral and dorsal routes during real-time reaching: evidence from imaging-guided multi-site brain stimulation. Behav. Brain Res. 300, 143–149 (2016).
Chen, Y. & Crawford, J. D. Allocentric representations for target memory and reaching in human cortex. Ann. N. Y. Acad. Sci. 1464, 142–155 (2020).
Wolbers, T. & Wiener, J. M. Challenges for identifying the neural mechanisms that support spatial navigation: the impact of spatial scale. Front. Hum. Neurosci. 8, 571 (2014).
This work was supported by the DFG grant FI 1567/6-1 ‘The active observer’ and by ‘The Adaptive Mind’, funded by the Excellence Program of the Hessian Ministry for Higher Education, Research, Science and the Arts.
The authors declare no competing interests.
Peer review information
Nature Reviews Psychology thanks the anonymous reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Fiehler, K., Karimpur, H. Spatial coding for action across spatial scales. Nat Rev Psychol 2, 72–84 (2023). https://doi.org/10.1038/s44159-022-00140-1