02 February 2024

Disentangling the brain’s symphony of the senses

In a groundbreaking study, UvA-scientists, including Matthijs Oude Lohuis, a Postdoc researcher currently working at the Champalimaud Foundation, have delved into the mystery of how our vision is influenced by events happening in other sensory modalities such as hearing, and by body movements. The study, published in Nature Neuroscience, marks a significant advancement in the field of neuroscience by showing how these ‘cross-sensory’ signals have an effect on an area of the brain classically thought to be processing only visual information.

Disentangling the brain’s symphony of the senses

Whenever we look at the world in front of us, light falls onto our retina and is processed in a series of steps in the brain. The primary visual cortex is the first stage in the cerebral cortex, where visual information is processed. Nevertheless, neurons here do not only respond to patterns of light, but to sounds as well. A major question in the field, however, has been whether these responses truly represent auditory information, or are instead modulations due to body movements elicited by the sounds.

Responding to sounds

To find out, researchers had a look at the way a mouse would respond to hearing certain sounds. “When we looked closely at the face of a mouse right after the sounds were played, we observed whisking, snout and eye movements. We know that these movements also activate the brain, including the primary visual cortex. This presented a puzzle: is sound-evoked activity present in the visual cortex because of the sounds we played, or because of motor reactions to the sound?”, says first author Matthijs Oude Lohuis, who was a PhD student at the UvA at the time and is presently a Champalimaud Research postdoc.

Different neurons, different neural circuits

The authors addressed this by relating the activity of individual neurons in the primary visual cortex to different sounds and body movements. They also silenced the auditory brain regions to observe the effects on the visual system. “A clear picture emerged: sound-evoked activity in the visual cortex consists of two signals: an auditory and a behavioural component”, says Umberto Olcese, a senior researcher in the team. “Each component has its own characteristics. The sound component comes from the auditory regions and is extremely fast. The body movements elicited by the sounds also evoke activity in the visual cortex, but this activity starts slightly later. The two components engage different neurons in different neural circuits.

Resolving an important debate

The fact that different neurons represent visual, auditory and motor information means that even when visual and auditory stimuli occur simultaneously, the information is represented alongside one another. “These findings resolve an important debate about the nature of cross-sensory signals in the brain” says Cyriel Pennartz, senior researcher in the team.

“Understanding these interactions is fundamental to a deeper understanding of the mechanisms governing sensory processing and unravel the mysteries of perception. In fact, one theory of consciousness posits that cross-sensory interactions are key to having qualitatively different experiences, and these findings are in line with this theory. Our brain makes sure that different senses can collaborate with and reinforce each other, but we also don’t want the senses to become totally mixed up. Our findings suggests how this may happen.”

Read the full publication here. 

Text adapted from the press release text prepared by UvA.
Please wait...