Real-time subtitle for the hearing impaired in augmented reality

Abstract

This master thesis aims to provide an introduction to the state of the art of some currently very promising technologies such as speech recognition, speech to text, augmented reality, face detection, face tracking, face recognition and sound localization. The thesis provides a brief overview of the aforementioned technologies in the chapter named Technical Basics. This also allows persons from other fields of expertise to read and understand the thesis. Furthermore, the thesis provides an insight to the current situation of deaf and hard of hearing persons in Austria and explains how the aforementioned technologies can be used in order to aid hearing impaired people in their daily lives. The master thesis furthermore comprised the development of a system prototype able to generate and present subtitles in real time in an augmented reality environment. Additionally, multiple visualization possibilities are provided for the generated subtitles. The implemented systems concept is also explained in this thesis, offering an overview of all involved hardware and software components. Then the implementation process of the developed system is explained in detail, while possible encountered limitations are presented and discussed, as well as identified solutions. The design and implementation process of the visualizations are explained and described as well. However, this master thesis consists only in part of the implementation of a system prototype aiming to ease the participation in meetings of deaf and hard of hearing persons. The second part of the thesis comprises the design and execution of a user study, during which deaf and hard of hearing users could test the implemented system prototype. The user study was organized in order to evaluate the implemented system. At the end of the thesis the extracted results are being analyzed and presented, followed by the conclusion as well as some suggestions for future work.