Brain Imaging Reveals What You’re Watching
Researchers develop an fMRI-based model to reconstruct moving images that people are seeing.
Scientists are a step closer to constructing a digital version of the human visual system. Researchers at the University of California, Berkeley, have developed an algorithm that can be applied to functional magnetic resonance imaging (fMRI) imagery to show a moving image a person is seeing.
Neuroscientists have been using fMRI to study the human visual system for years, which involves measuring changes in blood oxygen levels in the brain. This works fine for studying how we see static images, but it falls short when it comes to moving imagery. Individual neuronal activity occurs over a much faster time scale, so a few years ago the researchers behind the current study set out to devise a computer model to measure this instead. The study shows that this new approach is not only successful but remarkably accurate.
The study, which appears in Current Biology this week, marks the first time that anyone has used brain imaging to determine what moving images a person is seeing. It could help researchers model the human visual system on a computer, and it raises the tantalizing prospect of one day being able to use the model to reconstruct other types of dynamic imagery, such as dreams and memories.
Moving image: The image on the right was reconstructed from an fMRI image captured while the subject watched the video clip on the left.
Credit: Shinji Nishimoto, An T. Vu, Thomas Naselaris, Yuval Benjamini, Bin Yu & Jack L. Gallant