THE EFFECT OF SUBTITLES ON FILM UNDERSTANDING Pamela GRIGNON, Jean-Marc LAVAUR & Nathalie BLANC University Montpellier 3 Department of Psychology Route de Mende 34199 Montpellier Cedex 05 France [email protected] [email protected] [email protected]

Contact address for presenting author University Montpellier 3, Route de Mende, 34199 Montpellier Cedex 05 FRANCE [email protected]

Presentation Preference Preference for a poster presentation Abstract The aim of this study was to assess the effect of the subtitles in film understanding. We compared three versions of a film sequence (i.e., dubbed, subtitled, and original versions). The memory for both the visual elements and the dialogues of the sequence was evaluated, but also the inferences that were generated. One of the most important results was that the more difficult to understand the film sequence was, the more the viewers generated inferences. (75 words)

THE EFFECT OF SUBTITLES ON FILM UNDERSTANDING The comprehension of a film depends on the processing of various types of information viewers simultaneously encountered. Both visual elements (e.g., framing, montage, mise en scène, make-up, lighting …), sound elements (i.e., non-linguistic information such as noise or music), and linguistic elements (i.e., dialogues) participate to the understanding of a film. It is well admitted that visual elements play a crucial role in film understanding (see Magliano, Dijkstra & Zwaan, 1996) and more specifically in the generation of predictive inferences. Moreover, it has already been demonstrated that visual elements help viewers in detecting temporal and spatial shifts in the situation displayed (Magliano, Miller & Zwaan, 2001). The influence of nonverbal communication conveyed by the characters present at the screen could favor the comprehension of a second foreign language (presented orally). Indeed, Sueyoshi and Hardisson (2005) demonstrated that the same verbal message is better memorized when the framing allows to see both the face and the body of the person on the screen, rather than his/her face only. Not only viewers spontaneously use visual elements to understand a film, but they also automatically process the subtitled that appear on the screen. D’Ydewalle, van Rensbergen, Pollet (1987) showed evidence for the fact that reading subtitles is an automatic behavior in film understanding. Indeed, these authors observed that immediately after the appearance of the subtitles on the screen, viewers focused their attention on them, whatever the degree of knowledge they had on the film-spoken language. This phenomenon appeared even when the audiotape and the subtitles were in the mother tongue of the viewers (D' Ydewalle et al., 1987). It has also to be mentioned that the reading speed of the subtitles is not sensitive to viewers’ differences in watching subtitled films. Thus, participants less accustomed to watch subtitled films do not read the subtitles more slowly than participants who are used to watch subtitled films (D' Ydewalle et al., 1991). In line with these previous works, the aim of this study was to test whether the presence of subtitles would enhance the comprehension of a film sequence but, in the same time, would disrupt the memory for its visual elements. In other words, the question was to know whether the comprehension of a film sequence would vary as a function of its mode of presentation: with or without subtitles, with or without the presence of the mother tongue of the participants.

The sequence we selected corresponded to the beginning of the movie Lolita by Stanley Kubrick. We compared three different versions of this sequence. Thirty persons participated in this experiment. The participants’ knowledge of English language had been evaluated by their teacher (they were high-school students). All of our participants were of an equally English level of understanding. The participants were either provided with the dubbed version in the mother tongue (i.e., French), or with the original English version with French subtitles or with the original English version without any subtitles. Participants were invited to watch the selected sequence. Then, they were provided with a multiple-choice questionnaire that was composed of 48 questions. We distinguished three types of questions: 16 relied on the visual elements of the sequence, 16 dealt with the dialogues of the sequence and 16 were relative to inferences viewers were supposed to generate to understand the film sequence. Thus, with the questionnaire, we assessed the memory for both the visual elements and the dialogues of the sequence, but also the comprehension of the sequence through the inferences that were generated. We formulated three main predictions. First, we predicted that the memory for visual elements would be better in the original and dubbed versions than in the subtitled version. Second, we expected that the memory for the dialogues of the sequence would be better in the dubbed and subtitled versions, compared to the original version. Third, the comprehension of the sequence, evaluated through the inference judgments, would be better in the dubbed and subtitled versions, compared to the original version. We compared the performance the three groups obtained to the questionnaire. First, it appeared that the dubbed and subtitled versions lead to better performance than the original version (p < .01). Second, we noticed that the number of correct responses was the highest for the questions dealing with situational inferences, decreased for the questions on the dialogues and was the lowest for the questions about visual elements (p < .01). More importantly, the type of the versions of the film sequence (i.e., dubbed, subtitled, original) significantly interacted with the type of questions (i.e., visual elements, dialogues, situational inferences) (p < .01). Indeed, whereas the questions dealing with the dialogues gave rise to better performance than the questions focusing on visual elements in the subtitled version, the reverse pattern of response appeared for the dubbed version, no difference being observed in the original versions. Thus, the memory for visual and dialogue elements of the film sequence depended on the version viewers were provided with. Finally, it appeared that, unexpectedly, most of the participants who were provided with the original version of the sequence

answered more accurately to the inference questions compared to the two other types of questions. Overall, we interpreted these results as following: the more difficult to understand the version of the film sequence was, the more the individuals attempted to generate inferences.

References D’Ydewalle, G., van Rensbergen, J., & Pollet, J. (1987). Reading a message when the same message is available auditory in another language: The case of subtitling. In J. K. O’Regan & Lévy-Schoen (Eds.), Eye Movement: from physiology to cognition (pp. 313321). Amsterdam: North Holland. D’Ydewalle, G., Praet, C., Verdaillie, K., & van Rensbergen, J. (1991). Watching subtitled television: Automatic reading behaviour. Communication Research, 18, 650-666. Magliano, J., Dijkstra, K., Zwaan, R. A. (1996). Generating predictive inferences while viewing a movie. Discourse Processes, 22, 199-224. Magliano, J., Miller, J., Zwaan, R. A. (2001). Indexing space and time in film understanding. Applied Cognitive Psychology, 15, 533-545. Sueyoshi, A., & Hardisson, D. (2005). The role of gestures and facial cues in second language listening comprehension. Language Learning, 55(4), 661-699.