A study published in mid-April in the journal Nature Communications by Leopoldo Petreanu and his team, from the Cortical Circuits lab at Champalimaud Research, concludes that sensory processing by the visual cortex’s is not purely visual. More to the point, the study shows that, right from the early stages of sensory processing, the visual cortex integrates information from other sensory modalities, such as sounds.
“What we wanted to see”, says Petreanu, “was whether the auditory cortex sends spatial information signals to the visual primary cortex. We recorded directly from the axons – the nervous fibers – going from the auditory cortex to the visual cortex. And we found that they carry information about the properties of the sounds, such as frequencies and such, but also about the location from where the sound is coming. Our results show that the visual cortex, in the very early stages of visual processing, already has access to rich auditory information that contains spatial information.”
To do their experiments, the team put mice at the center of an array of speakers, spanning 120 degrees of the animals’ auditory space, and played sounds at different locations. The activity in the individual connecting axons was recorded using 2-photon calcium imaging, a technology for recording neuronal activity in the intact living brain.
One of the questions the researchers asked was whether the fibers that carry information about the sounds coming from a particular point in space target the neurons that “care” about that same location. In other words, are the auditory and visual cortex “mapped” onto one another?
The answer was no. The sound localization signals projected from the auditory to the visual cortex are non-topographic. “We didn't find any particular spatial correspondence”, Petreanu points out. There are other places in the brain, specifically the superior colliculus – a multisensory midbrain structure that integrates visual, auditory, and other spatial information – where the different sensory maps are aligned, but this is not the case in the primary visual cortex.
“Nobody had tested what happens in the visual cortex”, says Petreanu. A first conclusion from the new study is that the signals that the auditory cortex projects to the visual cortex are mainly auditory signals, even though the targeted area is a visual area. A second conclusion is that these signals contain spatial information. And a third important conclusion, as already mentioned, is that, contrary to other structures of the brain, that spatial auditory information is not topographically matched to the spatial “preferences” of the visual neurons it targets.
“We were expecting the opposite”, explains Petreanu, “because although auditory neurons have very different properties from visual neurons, they have one thing in common: location. So this lack of spatial mapping is very strange. But of course, this is just the first study.” Be as it may, this strongly suggests, according to him, that the integration of visual and auditory spatial information is different in the cortex – and that it probably has a different function too.
Visual spatial information is dominant over auditory spatial information in all species. That's why, for example, a ventriloquist will fool us, making us think that speech is coming from the moving mouth of his puppet and not from his own mouth. This suggests that the visual cortex probably does not use the spatial information from the auditory cortex to pinpoint the precise spatial position of a sound. And perhaps, that the spatial information coming from the auditory cortex is more about the global scene than about precise sound location.
What would the function of these connections be? “It could be to make a sense of the whole scene” speculates Petreanu. Sort of like being able to say: ‘If it looks like a jungle and sounds like a jungle, then I'm in a jungle’.” Even though the maps are not aligned, the brain might come to need the spatial information from the auditory cortex to figure out its position relative to its general surroundings.
“Clearly, the visual cortex has a lot of information available about the location of sounds, and maybe it needs to make connections with the location of sounds to predict where the visual signals are going to lie in the future”, Petreanu continues. “So the object, the sound, the object that gave rise to the sound might align in the eye after we make a movement”, he says.
Auditory and visual signals are not in the same coordinate system. Auditory space is defined by the head’s position, whereas visual space is eye-based, because the eyes can move independently of the head.
Also, since audition is “faster” than vision (you can hear a sound before you see the object), the auditory cortex can know where an object is before the visual cortex does. This could mean that the auditory cortex can predict where the image of an object is going to fall on the retina of the eyes a few moments later.
“After you integrate head position, eye position and maybe future movements, that can help an animal know beforehand what it should expect in his visual system”, says Petreanu. And he exemplifies: “If you hear barking outside your field of vision, and then you move your head towards that sound, you expect to see a dog”. So, though the signals seem not to be aligned, they might become aligned after the integration of future and postural changes of head and eye. “All this is very speculative”, notes Petreanu. “The function of this process remains to be understood.”
But one thing is certain, he concludes: “the visual cortex integrates information about the location of sounds with visual information.”.
Original paper here.
Image caption: In yellow, fibers from the auditory cortex in the visual cortex. Credits: Camille Maz
Text by Ana Gerschenfeld, Health & Science Writer of the Champalimaud Foundation.
