Cognitive Processes behind the use of Gestures in Consecutive Dialogue Interpreting.

Consecutive dialogue interpreting consists in bidirectional oral translation provided for users who do not speak the same language. It is the most frequent form of linguistic mediation provided in public service interactions with migrant users who do not yet master the language of the host country. Despite a seemingly language-centered character of interpreter-mediated encounters, the act of interpreting is described as a multimodal and embodied cognitive activity where gestures facilitate participatory meaning-making processes and help to coordinate turn taking. The study aims at investigating how gestural production of interpreters influences their cognitive load and the users’ satisfaction from the interpreting performance. The research design is based on non-invasive experiments involving students of interpreting departments who perform mock dialogue interpreting tasks resembling interactions in medical, police and administrative settings. Study 1 conducted onsite in Warsaw is focused on collecting psychophysiological data such as electroencephalogram (EEG) and Hear Rate Variability (HRV) of the interpreters, parameters known to indicate cognitive load, and cross-examining them with behavioral data gathered in video recordings. Study 2 conducted abroad (France, Spain) is based on remote interpreter-mediated interactions which are video recorded and followed by questionnaires and self-reports helping to examine subjective indicators of the cognitive load (e.g. level of fatigue, perceived level of difficulty of the task) and the appraisal of the interpreting performance delivered with or without spontaneous co-speech gestures.

Project Results

In spontaneous speech, people use not only words and different voice qualities to communicate. They also produce hand gestures to convey information about forms and shapes of the described objects, employ pointing to indicate directions, or reach towards the interlocutor to show that it is their time to respond. Gestures are omnipresent in human dialogue and constitute a precious tool of establishing a common conceptual ground and co-constructing the meaning of discussed ideas across speakers. The situation is no different in interpreter-mediated interactions, where speakers of two different languages, for instance a doctor and a foreign patient, communicate through the means of an oral translator called an interpreter. The interpreter working in consecutive dialogue mode listens to the source speech in one language and a couple of seconds later renders its meaning in another language so that the users can fully comprehend each other. Public service interpreters who typically use this mode often work in high-stake situations such as medical consultations, asylum hearings or police interrogations. They are obliged to restitute the meaning of the source speech as completely and accurately as possible. However, if we consider speech as a multimodal source of information, combining, verbal, vocal and visual cues, does it mean that interpreters should also repeat gestures of the speakers? 

On the one hand, the field of Gesture Studies shows that using gestures helps to remember difficult terms or maintain fluency when we speak a foreign language. On the other hand, in the course of training in consecutive interpreting, students are advised to limit their gesture production in order to “look more professional”. If gesture inhibition is an unnatural condition for speech production, does it create higher cognitive load in interpreters, making their work slower and more tiring? In parallel, do users of oral translation services indeed judge interpreters who frequently gesture as less professional than their non-expressive peers?

We conducted a study on 57 students of Master Degrees in Interpreting and Translation departments of the University of Warsaw – Poland, Université Paris Cité – France and Universitat Autònoma de Barcelona – Spain. The preliminary results of the project show that trainee interpreters present longer reaction times and higher scores in self-evaluation of the cognitive load when they cannot use their hands to perform spontaneous gestures, meaning that this artificial constraint makes their task more exhausting and time-consuming. To corroborate these findings, we collected video recordings of the experimental interpreting sessions that will be employed to further study gestures used by different interpreters in English – Polish, – French, – Spanish language combinations. Preliminary qualitative analysis of this multimodal corpus suggests that interpreting trainees mirror gestures of the speakers even when these speakers are only presented in video recordings and cannot benefit anyhow from this additional embodied information. Additionally, in order to take a look inside the black box of the interpreter’s mind, we collected electroencephalogram (EEG) and electrocardiogram (ECG) of the participants to investigate electrical activity of their brains and hearts during the interpreting tasks. These data will help us see if the interpreter’s body shows signs of increased mental effort and stress when the participants are instructed to refrain from gesture production during interpreting.

What if we look at the interpreting process form the perspective of a user of interpreting services? If a particular interpreter consciously moderates their own gesture production by keeping their hands folded on the table, in line with guidelines received during university training, will that guarantee that the clients will evaluate them as more professional than their peers who vividly move their hands during interpreting assignments? To test that, we asked our participants to assess 2 video recordings of interpreting performed in English-Turkish language combination, thus using one language that the raters fully comprehend and another – that they do not understand at all. This design ensured us that the raters will monitor visual cues the same way as clients do. Video recordings presented the same interpreter at work, the only difference being that in one of the videos they gestured extensively and gesturally aligned with the speakers, and in the second one they kept their hands still on the table. Our respondents received previous training in interpreting and were familiar with the rule that professional interpreters ought to limit their gestures. Nevertheless, the questionnaire responses from all cohorts (Warsaw, Paris, Barcelona) show that despite being aware of this rule, the respondents would still evaluate as more professional the interpreter who accompanied their target speech renditions with gestures and mirrored gestures of the speakers. Thus, the findings shed light on an intrinsic incoherence between pedagogy and practice of dialogue interpreting.

The project was presented at 7 international conferences in the fields of public service interpreting, linguistics and cognitive translation studies in Madrid, Brussels, Düsseldorf, Santiago de Chile, Lima, Nijmegen and Poznan.

Funding

The project is implemented thanks to the Marie Sklodowska-Curie Actions Fellowship awarded to Dr Monika Chwalczuk and hosted at the Institute of Psychology, Polish Academy of Sciences. The individual grant for Research and Innovation is part of the PASIFIC Program supported jointly by the Polish Academy of Sciences and the European Commission with a Marie Sklodowska-Curie Actions grant.