Main

The present study is based on a reanalysis of data from the first seven subjects in ref. 1, where details of the experimental procedure may be found. Subjects pressed a key with the left or right thumb to decide whether digits presented visually every 14 seconds were larger or smaller than 5. On each trial, seven fMRI brain volumes sensitive to brain oxygen-level dependent (BOLD) contrast were acquired on a three-Tesla Bruker imager. The repetition time (TR) of two seconds was sufficient to monitor the rise and fall of blood flow in task-related areas on a single-trial basis2. Four separate blocks of 32 trials were run, each preceded by a single training trial whose data were discarded. Thus, each subject contributed a total of 128 trials or 896 volumes of brain activation, except for one subject with only 96 trials and 672 volumes.

Planned contrasts for left-hand versus right-hand movement identified a network of areas known to be involved in motor control, comprising contralateral precentral cortex, contralateral supplementary motor cortex and ipsilateral cerebellum. Of these, the left and right precentral activations were found in all subjects and were highly significant (Z-score > 6). In addition, their coordinates were insensitive to the details of the statistical model used. These regions were therefore selected for further analysis.

After normalization, the time series of the left and right activations were subtracted to yield a lateralized motor preparation index ( Fig. 1). To predict motor responses, we computed the cross-product of the seven values of this index recorded on each trial with a phasic activation function, identical for all subjects, which had a Gaussian shape, a peak delay of 5 seconds and a standard deviation of 1.4 seconds. (Results were largely insensitive to this choice of parameters.) The classification of trials according to the sign of the lateralized motor preparation index correlated extremely well with the actual motor response made by the subject. There was a 97.2% agreement between the inferred and the actual response overall, with a range of 93.8% to 100% across the seven subjects, which is considerably higher than chance alone would predict (all χ2 (1 d.f.) >96, p < 10–21). Of 864 trials overall, only 24 were incorrectly classified by our brain-activation measure. Even on error trials, the subject's actual response was correctly inferred (15/17 = 88.2% correct ; χ2 (1 d.f.) = 9.94, p < 0.002).

Figure 1: Single-trial motor behavior can be inferred from fMRI signals.
figure 1

Top, left and right precentral cortical regions used for behavioral inference in an individual subject. Areas were identified with SPM96 software (http://www.fil.ion.ucl.ac.uk/spm) by correlating the functional images with waveforms derived from the known series of hand movements, taking into account the lag and shape of the hemodynamic response. Images were thresholded at p = 10–4, corrected for multiple comparisons across the brain volume to p < 0.05. Bottom, time course of the lateralized motor preparation index derived from the activation profiles of those brain regions. We extracted the time series {L(t)} and {R(t)} of the fMRI signal averaged across all voxels within the left and right sensorimotor clusters. These time series were normalized by subtracting their mean and dividing them by their standard deviation, and were then subtracted to yield the lateralized motor preparation index: LMPI(t) = R(t)–\(\overline{ R}\) / σR − L(t)–\(\overline{L }\) / σL. As shown here on a representative sample of 21 trials, almost all motor responses (L, left; R, right) were accompanied by a corresponding positive or negative peak of the LMPI.

Full size image

The above analysis might be criticized as being partly circular, because we first identified the left and right motor cortices based on their correlation with response side, and then used the time course of these areas as a predictor of response side. To obviate this criticism, we therefore re-ran the same analysis by first using an independent block of 32 trials to identify each subject's left and right motor cortices, and then running the prediction analysis on the remaining three blocks (96 trials). The prediction rate was still very high, averaging 90.6% correct with a range of 79.2–100% across subjects (all χ2 (1 d.f.) > 32.7, p < 10–8 ).

A previous attempt3 to predict behavior from physiological recordings of brain function used a sophisticated analysis of covariance across multiple channels in the electro-encephalogram (EEG) to achieve ~65% success in predicting the accuracy of a motor gesture. Mixed prediction results have also been obtained by other groups using EEG or magneto-encephalography (MEG)4,5,6. Our results, by contrast, suggest that fMRI has a greater signal-to-noise ratio than EEG or MEG and can provide estimates of brain activation that correlate with behavior on a single-trial basis with high accuracy.

A tight correlation between cerebral activity and behavior underlies the logic of all brain-imaging experiments. Reliably measuring this correlation in single trials, however, opens up new possibilities. Our work reverses the usual direction of most brain imaging experiments, in which the subjects' known behavior is used to throw some light on their pattern of cerebral activation. New experiments may now be envisioned in which the measured cerebral activity helps understand which behavioral or mental processes were occurring on any given trial. At present, inferring mental events from brain images is limited to experimental conditions with well defined trials, precisely known timing, a limited number of events and a cooperating subject. Yet within those limits, we see no reason why it could not be extended to the monitoring of covert mental events such as visual and motor imagery, internal speech, emotions or decision making. Several brain-imaging experiments have already investigated the neural correlates of covert mental events. For instance, distinct brain activity patterns have been observed when subjects mentally imagine objects at variable sizes7, listen to a known or unknown language8 or talk to themselves using their first or their second language9. That such private aspects of mental life can now be measured may ultimately raise important practical and ethical issues.