This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Scopus Google Scholar Articles by Rilliard, A. Articles by Aubergé, V. Social Bookmarking                         What's this?

Multimodal Indices to Japanese and French Prosodically Expressed Social Affects

LIMSI-CNRS, Orsay, France, albert.rilliard{at}limsi.fr

Takaaki Shochi

Kumamoto University, Kumamoto Japan

Jean-Claude Martin

LIMSI-CNRS, Orsay, France

Donna Erickson

Showa Music University, Kawasaki City, Japan

Véronique Aubergé

GIPSA-Lab, Grenoble, France

Whereas several studies have explored the expression of emotions, little is known on how the visual and audio channels are combined during production of what we call the more controlled social affects, for example, "attitudinal" expressions. This article presents a perception study of the audovisual expression of 12 Japanese and 6 French attitudes in order to understand the contribution of audio and visual modalities for affective communication. The relative importance of each modality in the perceptual decoding of the expressions of four speakers is analyzed as a first step towards a deeper comprehension of their influence on the expression of social affects. Then, the audovisual productions of two speakers (one for each language) are acoustically (F0, duration and intensity) and visually (in terms of Action Units) analyzed, in order to match the relation between objective parameters and listeners' perception of these social affects. The most pertinent objective features, either acoustic or visual, are then discussed, in a bilingual perspective: for example, the relative influence of fundamental frequency for attitudinal expression in both languages is discussed, and the importance of a certain aspect of the voice quality dimension in Japanese is underlined.

Key Words: attitudes • audovisual prosody • French • Japanese • perception

Language and Speech, Vol. 52, No. 2-3, 223-243 (2009)
DOI: 10.1177/0023830909103171


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?