Encoding of three-dimensional structure-from-motion by primate area MT neurons

Abstract

We see the world as three-dimensional, but because the retinal image is flat, we must derive the third dimension, depth, from two-dimensional cues. Image movement provides one of the most potent cues for depth^1,2,3,4,5,6. For example, the shadow of a contorted wire appears flat when the wire is stationary, but rotating the wire causes motion in the shadow, which suddenly appears three-dimensional. The neural mechanism of this effect, known as ‘structure-from-motion’, has not been discovered. Here we study cortical area MT, a primate region that is involved in visual motion perception. Two rhesus monkeys were trained to fixate their gaze while viewing two-dimensional projections of transparent, revolving cylinders. These stimuli appear to be three-dimensional, but the surface order perceived (front as opposed to back) tends to reverse spontaneously. These reversals occur because the stimulus does not specify which surface is in front or at the back. Monkeys reported which surface order they perceived after viewing the stimulus. In many of the neurons tested, there was a reproducible change in activity that coincided with reversals of the perceived surface order, even though the stimulus remained identical. This suggests that area MT has a basic role in structure-from-motion perception.