Major
Business Administration
Anticipated Graduation Year
2024
Access Type
Open Access
Abstract
We aim to determine how multitasking in the form of reading subtitles while consuming content affects emotional and analytical comprehension, perception, and mental effort. With the rise of streaming platforms, content is easily accessible in many languages with or without subtitles. While older viewers report subtitles are distracting and require more concentration, younger viewers, being avid social media users who are used to dual-tasking, are more accepting of subtitles. Eye-tracking, facial expression analysis, and galvanic skin response combined with survey questions will explore several research questions regarding viewing TV show clips with or without subtitles in various languages. In a within subject study using 32 participants we evaluated the impact of subtitles on comprehension, perception, and emotional response. In a fully randomized experiment subjects were exposed to four videos across four conditions (2 (English, Hungarian) by 2 (Subtitles, No subtitles) design). The study found that Subtitles in a viewer’s native language with audio in the same language produces better content comprehension than all other video options, when watching content with English subtitles and English audio, viewers performed better on content perception than all other video options, lack of subtitles with audio in a viewer’s native language produced greater subconscious response than all other video options (gaze patterns, facial expressions, and galvanic skin response peaks), and subjects with large amounts of prior experience using social media have an increased capacity to perceive visual content in videos with subtitles.
Faculty Mentors & Instructors
Dr. Dinko Bačić, professor of information systems at Quinlan School of Business
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.
Assessing the Impact of Video Options on Comprehension, Perception, Visual Attention, and Effort Using Biometrics
We aim to determine how multitasking in the form of reading subtitles while consuming content affects emotional and analytical comprehension, perception, and mental effort. With the rise of streaming platforms, content is easily accessible in many languages with or without subtitles. While older viewers report subtitles are distracting and require more concentration, younger viewers, being avid social media users who are used to dual-tasking, are more accepting of subtitles. Eye-tracking, facial expression analysis, and galvanic skin response combined with survey questions will explore several research questions regarding viewing TV show clips with or without subtitles in various languages. In a within subject study using 32 participants we evaluated the impact of subtitles on comprehension, perception, and emotional response. In a fully randomized experiment subjects were exposed to four videos across four conditions (2 (English, Hungarian) by 2 (Subtitles, No subtitles) design). The study found that Subtitles in a viewer’s native language with audio in the same language produces better content comprehension than all other video options, when watching content with English subtitles and English audio, viewers performed better on content perception than all other video options, lack of subtitles with audio in a viewer’s native language produced greater subconscious response than all other video options (gaze patterns, facial expressions, and galvanic skin response peaks), and subjects with large amounts of prior experience using social media have an increased capacity to perceive visual content in videos with subtitles.