International workshop on audio-visual affective prosody in social interaction & second language learning

(AVAP 2015)

Bordeaux, 5 and 6 March 2015

Albert Rilliard

Cross-cultural comparison of socioaffective expressive strategies - paradigm and first results

LIMSI CNRS, Paris XI University

Speakers modulate their prosody according to their communication goals and their situation of interaction; these changes are, arguably, partly dictated by their sociocultural origins (Wichmann, 2002). Many studies have investigated these expressivities, and described various strategies in different languages (Uldall, 1960; Martins-Baltar, 1977; Fujisaki & Hirose, 1993; de Moraes, 2008; Shochi et al., 2009). The labels used to describe such prosodic performances do vary from one language to the other, and the contexts of occurrence of similar labels may also vary, rendering the comparison of prosodic changes uncertain (Wierzbicka, 1992). It is thus difficult to compare the strategies recorded in different cultures, under these conceptual and contextual variations. To overcome this difficulty, a recording paradigm was developed, that bypass the limitations of label translation, aiming at comparing the prosodic strategies of speakers of different cultural origins. The present work focuses on USA English attitudes, as produced by L1 and L2 speakers (from Japan and France). L1 listeners then rate the quality of the audio-visual performances, extracted from dialogs implying given communication goals. A subset of the best expressions is then used for a perception test where 9 attitudes have to be recognized, in both audio and visual modalities. Results show that 6 categories of attitudes regroup the presented stimuli in coherent sets. The cultural origin of the speaker does affect the mean recognition performance, but only marginally the categorization of the expressions. To better understand the (dis)similarity across cultures observed in perception, the fundamental frequency and intensity of the stimuli is analyzed and discussed in light of two theoretical propositions: the frequency code (Ohala, 1994) and the effort code (Gussenhoven, 2004). The observations conclude to a main coherence of expressivity across language groups. Some differences still exist and are discussed. The case of the negative expressions of imposition (contempt, obviousness, irony) is analyzed to conclude that L1 speakers may follow a strategy compatible with the frequency code (and L1 listeners are expecting changes accordingly); on the contrary, L2 speakers do use the predictions of the Effort code in that situation – leading to perceived confusions in the audio-only modality. Differences for seduction or irony are discussed: the observed perceptual mismatches are linked to the expressive choices of speakers, linked to their cultural origin. These mismatches can generally be solved in bimodal presentations. The problems raised by irony are discussed but would need more analysis to propose a grounded explanation.

References:

Uldall, E. (1960). Attitudinal meanings conveyed by intonation contours. Language and Speech, 3(4): 223–234.

Fujisaki, H. & Hirose, K. (1993). Analysis and perception of intonation expressing paralinguistic information in spoken Japanese. ESCA Workshop on Prosody, 254-257.

Gussenhoven, C. (2004). The phonology of tone and intonation. Cambridge University Press.

Martins-Baltar M. (1977). De l’énoncé à l’énonciation: une approche des fonctions intonatives. Paris: Didier.

de Moraes, J. A. (2008). The pitch accents in Brazilian Portuguese: Analysis by synthesis. Speech Prosody, 389–397.

Ohala, J. J. (1994). The frequency codes underlies the sound symbolic use of voice pitch.
In Hinton, L., Nichols, J. & Ohala, J. J. (Eds.), Sound symbolism, Cambridge: Cambridge University Press, 325–347.

Shochi, T., Rilliard, A., Aubergé, V. & Erickson, D. (2009). Intercultural perception of English, French and Japanese social affective prosody, In S. Hancil (ed.), The role of prosody in affective speech, Linguistic Insights 97, Bern: Peter Lang, AG, Bern, 31-59.

Wichmann, A. (2002). Attitudinal intonation and the inferential process. Speech Prosody, 11–16.

Wierzbicka A. (1992). Defining Emotion Concepts. Cognitive Science, 6:539-581.