Music and speech are alike in that they use both facial and acoustic cues to engage listeners in an emotional experience. Regular concert-goers are used to seeing singers use expressive and often very dramatic facial expressions. Researchers at McGill University have now demonstrated that facial expressions are more important to convey emotions in music than in speech.
Researchers conducted an experiment where participants were offered recordings of short phrases (neutral statements, all of which were seven syllables long, such as 'children tapping to the beat' or 'people talking by the door'), which were then spoken or sung with a variety of emotions. The study participants were then offered these recordings in three different format i.e. either audio alone, video alone (with no sound), or full audio-video recordings. They were then asked to identify the emotions that the performers' intended to convey.
The researchers found that participants had a hard time recognizing the emotion based on the audio recording alone. However, once visual cues were added, the observers' understanding of the emotions the music sought to convey improved dramatically. They also noticed that participants were much better able to recognize emotion in speech, whether they were listening to audio alone, watching a video without sound, or seeing both at the same time. Therefore, the researchers believe that visual cues play a much more important role in the understanding of the emotions being conveyed by music than they do in the understanding of speech.
The study has been published in the Quarterly Journal of Experimental Psychology.