Reconstructing the Mind's Eye

Guest blog post by Simon. Simon contributes to the neurobonkers.com blog which will be following this research closely. You can read more on this post here and subscribe to neurobonkers here. You can also receive updates via Twitter, Facebook and Google+.

Scientists successfully reconstruct the mind's eye for the first time as a video output

A group at Berkeley has just published the first successful attempt to reconstruct colour video imagery from the mind using an fMRI brain scanner. The results are startling, encouraging and a little bit scary. The method used, called fMRI, is known for its high spatial resolution (3D imaging ability) but notoriously low temporal resolution (measurements with respect to time - effectively a slow shutter speed). In the past, this has been a barrier to research on the visual cortex because of the incredibly high rate of information processing in the visual system. However, scientists have recently developed a new MRI encoding method that allows for the modelling of brain activity in the visual system at a faster rate. In the experiment conducted by the Berkeley group, participants were shown 7,200 seconds of random colour video clips one time only, while their brains were scanned using the novel fMRI sequence. From these scans, researchers were able to create a "dictionary" of brain activity in the visual system.

After a dictionary of brain activity in the visual system was created, the participants watched a fresh unseen video from YouTube while undergoing a brain scan. This resulted in video outputs that resembled the new YouTube video shown to participants. The output appears as a collage of flickering pixels that reminds me of the paintings of prosopagnosia sufferer Chuck Close. The correlation between the videos shown to the participants and the output imagery in the collage-like videos (below) is pretty astounding especially when considering there is zero overlap between the clips used for calibration and the clips used to test the system.

The study authors suggest that the method used in this paper could eventually lead to the generation of video output from participants experiencing dreams or hallucinations. Watch this space! What was once a field reserved firmly for science fiction may fast be becoming a reality.

Listen to an NPR interview with the researchers here:

References:

Nishimoto S, Vu AT, Naselaris T, Benjamini Y, Yu B, & Gallant JL (2011). Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies. Current biology : CB PMID: 21945275