Aero-tactile integration in speech perception
Nature 462, 502-504 (26 November 2009)
Bryan Gick, & Donald Derrick
http://www.nature.com/nature/journal/v462/n7272/full/nature08572.html
Visual information from a speaker’s face can enhance or interfere with accurate auditory perception.
This integration of information across auditory and visual streams has been observed in functional imaging studies, and has typically been attributed to the frequency and robustness with which perceivers jointly encounter event-specific information from these two modalities.
Adding the tactile modality has long been considered a crucial next step in understanding multisensory integration. However, previous studies have found an influence of tactile input on speech perception only under limited circumstances, either where perceivers were aware of the task or where they had received training to establish a cross-modal mapping. Here we show that perceivers integrate naturalistic tactile information during auditory speech perception without previous training.
Drawing on the observation that some speech sounds produce tiny bursts of aspiration (such as English ‘p’), we applied slight, inaudible air puffs on participants’ skin at one of two locations: the right hand or the neck. Syllables heard simultaneously with cutaneous air puffs were more likely to be heard as aspirated (for example, causing participants to mishear ‘b’ as ‘p’).
These results demonstrate that perceivers integrate event-relevant tactile information in auditory perception in much the same way as they do visual information.
journalistic version:
Sense Of Touch Can Help Hearing, Study Says
November 27, 2009
http://www.npr.org/templates/story/story.php?storyId=120873368
The study builds on decades of research showing that the brain often uses visual information to augment hearing. That’s why people in a noisy room are more likely to understand someone if they can see the speaker’s lips.
“From our brain’s point of view, we can hear with our eyes,” says Bryan Gick, a professor of phonetics at the University of British Columbia in Vancouver. But he and his colleague Donald Derrick wanted to see whether hearing could also be influenced by our sense of touch.
keyword: multisensory development
related:
https://franzcalvo.wordpress.com/2013/08/26/sight-over-sound-in-the-judgment-of-music-performance