Psychoacoustic Cues in Speech and Music
May 17, 2013, Music Meets Science Forum, SwissNex, San Francisco—Eduardo Coutinho from the University of Geneva is working to extract the features that evoke emotions in music and voice. Eventually they want to predict which sonic features affect specific emotional responses.
In speech, we can detect changes in tone and pitch to understand feelings without understanding the words. In music, the patterns and sound associations from shared experiences seems to have similar responses, but are uncharted.
Therefore, research is ongoing to identify the features and basis for emotional evocation from music, predict responses, and evaluate the nature of the interactions. In experiments with music and speech samples, test subjects were asked to move a cursor in a field with axes of valence—pleasant or unpleasant—and arousal—calm or excited. This continuous measurement demonstrated changes in emotion and trajectories of the changes. Subjects were also fitted with other sensors to measure physiological parameters. After the audio testing, test subjects were asked to identify their feelings with words.
The models for relating sonic features and the expression of emotions are emerging. A non-linear regression model uses a test, training, evaluation sequence to develop predictions. Some of the parameters are loudness, tempo, pitch, timbre, and dissonance. It appears that six parameters may be sufficient for prediction.
In speech, 84 percent of the variances in arousal measures and 60 percent of valence correlate to distinct areas of the grid. For music the numbers are 75 percent for arousal and 43 percent for valence. A partial reason for the loser numbers in music is that music seems to invoke more and varied emotions than speech. The low-level parameters can provide cues that can predict many emotional responses, but the dynamic details and complex responses mean that we still don’t know the why of the responses.
The distinct responses imply that the information can be used for diagnoses in psychiatric work. largely different responses from the common profiles could indicate problems. The human–computer interfaces could be enhanced to provide other information once the specific brain areas and emotional responses are cataloged.
