D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . VR Bridges Simulation von unebenen Flächen in VR DIPLOMARBEIT zur Erlangung des akademischen Grades Diplom-Ingenieur im Rahmen des Studiums Visual Computing eingereicht von Bálint István Kovács Matrikelnummer 01227520 an der Fakultät für Informatik der Technischen Universität Wien Betreuung: Assoc.Prof. Priv.Doz. Mag. Dr. Hannes Kaufmann Mitwirkung: Univ.Ass. Khrystyna Vasylevska, BSc MSc Wien, 30. April 2020 Bálint István Kovács Hannes Kaufmann Technische Universität Wien A-1040 Wien Karlsplatz 13 Tel. +43-1-58801-0 www.tuwien.ac.at D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . VR Bridges An Approach to Simulating Uneven Surfaces in VR DIPLOMA THESIS submitted in partial fulfillment of the requirements for the degree of Diplom-Ingenieur in Visual Computing by Bálint István Kovács Registration Number 01227520 to the Faculty of Informatics at the TU Wien Advisor: Assoc.Prof. Priv.Doz. Mag. Dr. Hannes Kaufmann Assistance: Univ.Ass. Khrystyna Vasylevska, BSc MSc Vienna, 30th April, 2020 Bálint István Kovács Hannes Kaufmann Technische Universität Wien A-1040 Wien Karlsplatz 13 Tel. +43-1-58801-0 www.tuwien.ac.at D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . Erklärung zur Verfassung der Arbeit Bálint István Kovács Táncsics u. 23, 9400 Sopron, Ungarn Hiermit erkläre ich, dass ich diese Arbeit selbständig verfasst habe, dass ich die verwen- deten Quellen und Hilfsmittel vollständig angegeben habe und dass ich die Stellen der Arbeit – einschließlich Tabellen, Karten und Abbildungen –, die anderen Werken oder dem Internet im Wortlaut oder dem Sinn nach entnommen sind, auf jeden Fall unter Angabe der Quelle als Entlehnung kenntlich gemacht habe. Wien, 30. April 2020 Bálint István Kovács v D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . Acknowledgements First and foremost, I would like to express my deepest gratitude to my advisors Khrystyna Vasylevska and Prof. Dr. Hannes Kaufmann for their patience, guidance and continuous support towards my work on this thesis. I would also like to thank the Interactive Media Systems Group for allowing me to work on such a current, real-world application topic and providing the infrastructure and equipment to realize this idea. Furthermore, I’m grateful towards the colleagues from the IMS for their expert feedback during development, and towards the participants of the user studies for their time invested into providing data for this project. I would like to thank my beautiful wife Judit for taking care of everything while I devoted my time to this project, and my mother, mother-in-law and two lovely daughters Gréta and Hédi for putting up with me during this period. vii D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . Kurzfassung Virtuelle Realität (VR) verspricht ein grenzenloses Erfahrungspotential. Durch technische Einschränkungen sind jedoch derzeitige VR-Erfahrungen oft in vielerlei Hinsicht begrenzt und nicht mit ihren Entsprechungen in der echten Welt vergleichbar. Begehbare gewölbt- unebene Oberflächen sind in der Realität allgegenwärtig, in VR aber kaum verwirklicht. Gleichzeitig ermöglicht VR aber die Veränderung und Manipulation von Wahrnehmung, und bietet dadurch Mittel zur Neuformung der Erfahrung. In dieser Diplomarbeit untersu- chen wir die Möglichkeit, begehbare gewölbte Oberflächen in VR durch multi-sensorische Sinnesreize zu simulieren. Wir erkunden die menschliche Wahrnehmung von Höhe und Neigung, und verwenden die gewonnenen Erkenntnisse in einem multi-modalen Ansatz, in dem visuelle Manipulationen, haptische und vibro-taktile Reize kombiniert werden. Unser Ansatz wird durch die Konstruktion von physischen Brückenrequisiten und die Entwicklung einer komplexen Softwareapplikation verwirklicht, die den Zufuhr der ver- schiedenen Sinnesreize an den Benutzer ermöglichen. Die Simulation wird mithilfe von zwei Benutzerstudien evaluiert, die jeweils auf eines von zwei Brückenrequisiten fokussie- ren. In den Studien evaluieren wir die Brauchbarkeit eines flachen und eines tatsächlich gewölbten Brückenrequisits für die Simulation von verschiedenen virtuellen Oberflächen- höhen. Die Daten, die im Laufe der Studien gesammelt werden, werden einer quantitativen und qualitativen Analyse unterzogen. Unsere Ergebnisse lassen darauf schließen, dass eine tatsächlich gewölbte Oberfläche die überzeugende Simulation von deutlich höheren virtuellen Oberflächen ermöglicht, als die physische Höhe der echten Oberfläche. Das haptische Feedback der gewölbte Oberfläche und die propriozeptiven Signale, die durch die tatsächlichen senkrechten Bewegungen entstehen, führen zu solchen Benutzerschätzungen von Oberflächenhöhe und -neigung, die näher an den Werten sind, die von den visuellen Reizen suggeriert werden. Die flache physische Oberfläche wird weniger realistisch empfunden und führt trotz simulierter visuellen Höhen- und Neigungssignalen zu Unterschätzung der Höhe und der Neigung. Für die Simulation von kleineren Unebenheiten kann das flache Brückenrequisit jedoch verwendet werden. ix D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . Abstract Virtual reality (VR) promises boundless potential for experiences. Yet, due to technical restrictions, current VR experiences are often limited in many ways and incomparable to their real-world counterparts. Walkable smooth uneven surfaces are inherent to reality but lacking in VR. At the same time, VR enables the alteration and manipulation of perception, offering tools for reshaping the experience. In this thesis, we explore the possibility of simulating walkable smooth uneven surfaces in VR via a multi-sensory stimulation approach. We examine human height and slant perception and incorporate our findings into a multi-modal approach by combining visual manipulations, haptic and vibrotactile stimuli. Our approach is realized by constructing physical bridge props and creating a complex software application to introduce multi-sensory stimuli to the user. The simulation is evaluated in two user studies, each focusing on one of two differently shaped physical bridge props. In the studies, we evaluate the feasibility of a flat and an upward curved prop for the simulation of different virtual surface heights. The data collected during the studies is subjected to a qualitative and quantitative analysis. Our results suggest that the use of a curved prop enables the convincing simulation of significantly higher uneven surfaces than the actual height of the prop. The haptic feedback of the curved surface and the proprioceptive cues of actual vertical traversal facilitate user provided height and slant estimations to be closer to the values suggested by the visual cues. The use of a flat prop is less realistic and leads to height and slant underestimations, despite the simulated visual height and slant cues. However, a flat surface might be still used to simulate indentations and protrusions with smaller height differences. xi D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . Contents Kurzfassung ix Abstract xi Contents xiii 1 Introduction 1 1.1 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Structure of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 Related Work 5 2.1 Real World Height and Slant Perception . . . . . . . . . . . . . . . . . 5 2.2 Presence and Embodiment . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Multi-Sensory Stimulation for VR . . . . . . . . . . . . . . . . . . . . 10 2.4 Locomotion and Redirection . . . . . . . . . . . . . . . . . . . . . . . . 14 2.5 Vertical Locomotion and Uneven Surfaces . . . . . . . . . . . . . . . . 17 3 Conceptual Design 23 4 Implementation 25 4.1 Physical Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.2 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5 Procedure Description 45 5.1 General Study Procedure . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.2 Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.3 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 6 Evaluation 53 6.1 User Study 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 6.2 User Study 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 7 General Discussion, Conclusion and Future Work 67 7.1 General Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 xiii D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 7.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 7.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 A Variable Height Physical Bridge Prop 73 List of Figures 79 Bibliography 83 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . CHAPTER 1 Introduction 1.1 Problem Definition The term virtual Reality (VR) describes a concept of artificial environments that are not existent in fact, but can be perceived in effect through sensory stimuli. The idea of immersing a person in an encompassing, simulated artificial world has long been a subject of interest for science-fiction narratives. The key enabling technologies, like 6 degrees of freedom head tracking and head-mounted displays with stereoscopic rendering capabilities have been proposed over 50 years ago [1]. Although research for training and industrial applications continued steadily, a new advent in VR research was ushered in by the release of commercially available, affordable enthusiast devices and systems, such as the Oculus Rift [2] and the HTC Vive [3]. VR enables the creation of virtual environments (VEs) unlimited in possibilities with regards to different conceptual dimensions, such as design, interaction or perception. However, one of the fundamental problems of VR is the gap between these unlimited possibilities of the virtual world(s) and the limited size of physical workspace. In the real world, we are used to the fact that natural locomotion inherently includes movement in the horizontal directions and in the vertical direction as well: walking up and down stairs, stepping onto roadside curbs, climbing hillsides, or crossing bridges. In the ideal VR setup, the freedom of movement is also unrestricted both in the horizontal and the vertical directions. In practice, however, most current VR setups employ magical locomotion metaphors for vertical traversal because of their ease of use and nonrestrictive quality. Natural movement is usually restricted to the horizontal surface in a small workspace. Different methods have been proposed to close the gap between the possibilities of VEs and the restrictions of physical workspaces with regards to enabling vertical traversal and simulating uneven surfaces. When designing methods for the purpose of introducing enabling vertical traversal in VEs, one major challenge lies in the trade-off between 1 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 1. Introduction maximizing the reachable virtual space and retaining the user’s immersion. Another difficulty is centered around considerations for the health ans safety of the user, as the introduction of actual physical unevenness into the workspace poses an increased risk of injury. Magical traversal metaphors, such as flying or teleportation, allow for VEs of unlimited size both horizontally and vertically, but are incomparable to experiences in physical reality. In contrast, natural locomotion metaphors, such as real walking, allow for a high degree of immersion and have been shown to have a positive influence on user’s sense of presence [4], spatial updating [5], search task performance [6], attention [7] and higher mental processes [8]. Consequently, methods employing natural locomotion metaphors are beneficial in application scenarios where maintaining immersion is desired. Several methods have been proposed before that ease restrictions on walking in a limited physical space: redirected walking [9], flexible and self-overlapping spaces [10, 11], specialized devices [12], etc. However, the vast majority of methods are designed for VEs where the accessible real and virtual environment only consists of a flat surface, limiting the possibilities of application scenarios. A smaller number of publications tries to tackle the problem of natural vertical locomotion and non-flat VEs. Some employ software-only solutions, such as camera view-port manipulations to simulate visual cues [13]. Others use specialized passive [14, 15, 16] or active [17, 18] devices to simulate haptic or proprioceptive cues. In this regard, multi-modal stimulation approaches are promising, as research suggests that stimulating different parts of the sensory apparatus simultaneously has a positive impact on immersion and the forming of memories [19]. However, although these techniques proposed for simulating non-flat VEs do in fact offer vertical locomotion, they mostly simulate different variations on small, incremental flat surfaces, such as stairs or curbs. Solutions for true smooth uneven surface simulation have hardly ever been published. In the scope of this thesis, we propose to explore the possibility of true uneven surface simulation with multi-sensory stimulation and real natural walking. The key research questions of the proposed thesis can be formulated as follows: • How can uneven surfaces be simulated effectively? • How does each sensory component of the stimulation contribute to the effectiveness of the simulation? • How much of a height difference can be simulated without breaking the illusion? Where are the thresholds of the simulation? 1.2 Structure of the Thesis In Chapter 2 we provide an review of publications from fields related to our research, from the basics of height and slant perception, through concepts of the sense of presence 2 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 1.2. Structure of the Thesis and embodiment in a VR context, to a summary of techniques we implement ourselves, such as multi-sensory stimulation, redirection, and simulation of uneven surfaces in VR. In Chapter 3 we define the stimulation factors we plan to investigate in this project. We outline our research goals and formulate our hypotheses. In Chapter 4 we give a comprehensive overview of the the implementation work we invested into creating physical devices and software applications that enable our research. We discuss the construction of the physical bridge props and the the design of our research application, including the main uneven surface simulation concept based multi-sensory stimulation, and the supplementary concepts of redirection, user interface design and data-acquisition. In Chapter 5 we provide detailed information about the user studies we have conducted. We describe the general study procedure common to both studies, the task participants were asked to perform and the measures we defined for the evaluation. In Chapter 6, we continue with separate reporting on the evaluation of each of the user studies, presenting setup particularities, result analyses and discussion. In the final Chapter 7, we provide a summary of the insights gained during our research, discuss additional ideas, new findings and propose further refinement possibilities. 3 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . CHAPTER 2 Related Work The following chapter contains an overview of literature related to different aspects of our research. In Section 2.1, we examine works on the mechanisms of human height and slant perception to understand how sensory input from different sources contributes to a coherent perception of spatial layout. In Section 2.2, we discuss two concepts connected to the perception of the relation between spatial layout and the self: the sense of presence and the sense of embodiment. In Section 2.3, we examine the application potential of the insights about spatial layout perception in a VR context. In Section 2.4, we provide an overview of redirection techniques. In the final Section 2.5, we review methods proposed for vertical locomotion in VR, with a special attention to techniques employing multi-sensory stimulation. 2.1 Real World Height and Slant Perception Human perception of (geographical) height and slant is an extensively researched field outside of the VR context. This publication corpus is relevant not only as a basis to how manipulations should be implemented, but also as a guide to how user feedback should be captured and evaluated. When discussing height and slant perception, an important distinction has to be made between the representation of geographical height and slant in conscious awareness and the performance of visually guided actions[20, 21]. Height and slant representation in conscious awareness denotes the mental concept about a specific height or slant. These are the measurements that can be acquired via asking e.g. study participants provide estimations. On the other hand, in the context of perceiving the height and slant of walkable surfaces, performance of visually guided actions denotes the ability to coordinate the movement of body parts to perform locomotion on a slanted, uneven surface. The visual characteristics of the surface guide the coordination of limb motions to perform 5 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2. Related Work Figure 2.1: Slant perception experimental views using real and virtual environments from Proffitt et al. [22] walking or running. The performance of visually guided actions can be captured by observation (e.g. watching study participants during task performance). The perception of slant is reliant on visual cues about surface characteristics. Related literature shows that people tend to overestimate slant in conscious awareness, independent from the viewing direction [22]. Even though the side view of a hill provides better cues about surface characteristics than a head-on view, there is still a gross overestimation of the slant. Proffitt et al. [22] studied this phenomenon in both real and virtual environments (see Figure 2.1). However, the magnitude of overestimation has been shown to be dependent on a number of factors. These factors include: • Physical state: Bhalla and Proffitt report that study participants perceived hills as being steeper when they were encumbered by a heavy backpack, were tired, in a state of low physical fitness, of an elderly age or in declining health [23]. • Mood: According to the findings of Riener et al., study participants in a sad mood reported higher slant that those in an excited/happy mood [24]. One possible explanation for this is a difference in attention between different moods. Another explanation is based around mechanisms of "affect as information", meaning that mood plays an information-like role in judgement situation that can not be explained by mood-dependent memory effects. In other words, judgments is passed from a "gut feeling" rather than mood-dependent memory search. Further theories provide physiological explanations, such as differences in blood glucose concentration. • Fear: Stefanucci et al. report that study participants judged the steepness of a hill higher when looking down from the top vs. looking up from the bottom, and even higher when being in a scared state of mind (standing on top on a skateboard) [25]. 6 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2.1. Real World Height and Slant Perception • Social support: Schnall et al. report that study participants accompanied by a friend reported lower estimated slant than others who were alone [26]. Even the recollections about a supportive person yielded similar results. While these factors can influence conscious awareness, the studies listed above unanimously report that visually guided actions remain unaffected. There is no observable difference in task performance, apart from physiological factors that are hard to control against, such as deterioration of motor functions due to old age. The research theories and findings relating to the influencing factors listed above are part of the encompassing theory of embodied perception [27]. Embodied perception is understood as the sensory apparatus biasing perception to influence behaviour. According to embodies perception, the perception of height, slant and other spatial layout characteristics does not only depend on visual cues, but also on the performance costs of associated actions. Conscious estimation of layout characteristics is influenced by a persons capability of navigating the environment in question. The majority of the reviewed literature subscribes to this interpretation. However, there exist alternate theoretical frameworks that question this understanding of perception. Firestone argues against the interpretation of experimental results, labeling embodied perception as a "wrong kind of theory" for understanding the processes involved in spatial perception [28]. While we are aware of the question raised about its validity, the concept of embodied perception still proves important in the research of phenomena related to height and slant perception. Embodied perception aids the understanding of factors that can potentially influence reported estimations. Although a less often researched topic, the perception of height has been shown to be governed by similar principles [29]. In conscious awareness, people overestimate height as well. The extent of the overestimation has been shown to be dependent on factors such as the following: • Fear: Clerkin et al. show that fear- and anxiety-inducing imagery has an influence on height perception [30]. People, who intentionally put themselves in an anxious disposition by picturing themselves falling, provided greater overestimation of height. Having acrophobia further impacted perception: participants with a preexisting fear of heights tended to overestimate height even more when subjected to anxiety inducing imagery. • Emotional state: Stefanucci et al. show that heightened emotional states can influence height perception [31]. States of both positive and negative arousal have been shown to yield an increase in reported height estimations. A heightened state of arousal influences attention, generally narrowing the focus of attention and steering attention to different (visual, auditory, etc.) cues. • Observer position: Stefanucci et al. report that the extent of overestimation depends on whether height is judged looking up from below, or looking down from above [29]. Looking down results in a more significant overestimation. 7 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2. Related Work Real world height and slant perception research discussed in this section also provides information about experimental apparatus used to capture participant feedback. Refer to Section 4.2.4 for a comparison of our slant measure apparatus and the devices proposed in the literature on real world slant perception. 2.2 Presence and Embodiment 2.2.1 Presence For the purposes of VR, the term presence denotes the sensation of "being there" in the virtual environment. Schuemie et al. provide a comprehensive overview of different attempted definition, based of different underlying perceptual or cognitive aspect [32]. They report and summarize several theories on the nature of presence in immersive VR. We propose the following selection of theories from their review as most relevant to our research: • Presence as non-mediation: Objects, events and the environment itself are perceived as if they were not, or not entirely mediated by technological means. In this regard, perception and presence is defined as the meaningful interpretation of the experience. • Exclusive presence: At any moment, users can only feel as being present in only one environment between the real, the virtual and the imaginary. Because of this, a high sense of presence in one environment requires a low sense of presence in the other two. • Ecological view: This view is based on the ecological theory of perception, consisting of three main concepts. – Situated affordances of the environment: The possibilities of actions are understood as characteristics of that environment [33]. – Perception-action coupling: Perception of an environment is dependent on possible actions that the perceiving organism can perform in it [32]. – Tool as functionality: Awareness of a tool is reduced to its function in the current task performance [34]. The ecological theories on the other hand promise a transferability of the findings of real-world height and slant perception studies. If perception is dependent on the observers (physical, emotional, etc.) state, the affordances of the environment will be interpreted accordingly. The success of our uneven surface simulation is intertwined with the maintaining of a high sense of presence in the user. Since the proposed approach employs multi-sensory stimulation, presence and perception as the meaningful interpretation of the experience [32, 35, 36] is a key concept, as different sensory cues have to be consolidated into a 8 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2.2. Presence and Embodiment single coherent experience. Of the different cues, the visuals can be considered mediated, while the haptic / proprioceptive cues are not. For a successful simulation, sensory consolidation from different conflicting sources still has to be reasonably possible to maintain exclusive presence. This requirements mandates a careful exploration and design of manipulations, with a special regard for thresholds. 2.2.2 Embodiment While the sense of presence is understood as "being there" in an environment (self- environment relationship), the sense of embodiment refers to "being there" inside a body (self-body relationship), that serves as a boundary between the environment and the self and obeys one’s will. Kilteni et al. provide a comprehensive summary of the research on the connection between the sense on presence and the sense of embodiment with regards to the VR context [37]. They distinguish three key concepts of the sense of embodiment: • Sense of self-location: The volume in space where the own body is perceived to be located. • Sense of agency: The experience of being able to perform motor actions according to one’s own intentions and conscious will. • Sense of body ownership: One’s self-attribution to a body as the source of the experienced sensations. Of the three concepts, the sense of self-location is the most relevant for the research presented in this thesis, as we can leverage the perception mechanisms of self-location in our multi-sensory stimulation approach. A full body representation (avatar) is not a hard requirement for self-location [37]. One of the factors in self-location is that space is partitioned into personal space (that the body occupies), peripersonal space (that is within arms’ reach) and extrapersonal space (beyond arms’ reach) by the brain. The border between peripersonal and extrapersonal space can shifted outward via the use of tools, leading to the concept of tool embodiment. Tool embodiment has been studied specifically in VR contexts. Alzayat et al. devised a quantitative measure for tool embodiment in VR, and showed that reliably working tools elicit tool embodiment without body representation [38]. This was especially the case when the physical tool prop (grabber) and its virtual representation were closely matched. Refer to Figure 2.2 to the participants view in their experimental setup. The findings of Alzayat et al. support the assumption that self-location is constructed from multi-modal/multi-sensory cues as reported by Kilteni et al. [37]. Although self- location highly relies on visual cues, vestibular and tactile signals also have been shown to be influencing factors. However, the consistency of different cues in a specific situation can have significant impact on self-location. One example of this is the rubber hand illusion, during which synchronized haptic and visual stimulation on corresponding spots between 9 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2. Related Work Figure 2.2: Participants’ view in the tool embodiment experiment from Alzayat et al. [38]. Note the lack of full-body representation, which is not a hard requirement for the sense of embodiment to form. real and rubber hands creates the illusion of feeling in the rubber hands [39]. This can cause stimulated body regions to be mislocalized, described with the term proprioceptive drift [37]. Meanwhile, the effect is inhibited if the multi-sensory stimulation is out of sync [37]. Furthermore, it has been shown that with congruent multi-modal stimulation, the location where the stimulus was observed can override the visual perspective of self- location [40]. In practice, this means that that the spatial location where the stimulation was seen has a high chance of being accepted as the actual position of occurrence, even if proprioceptive cues provide conflicting information. These findings show that through synchronous multi-modal/multi-sensory stimulation, self-location can be predictably manipulated, even if different sensory cues suggest impossible body-configurations. We can leverage the effects of tool embodiment on self-location by introducing location manipulations to handheld VR controllers. According to previous research reviewed in this subsection, manipulating the controller position to connect with a simulated virtual geometry at the same instant it connects with a differently shaped real physical props should allow to increase the effect of illusion. For this reason, we include controller position manipulation in our visual stimulation. Refer to Section 4.2.2 for implementation details. 2.3 Multi-Sensory Stimulation for VR In the project presented in this thesis, we employ multi-modal/multi-sensory stimulation in a VR-based simulation context. In this section, we further examine previous work on how information from multiple sensory sources is combined into a single experience. We 10 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2.3. Multi-Sensory Stimulation for VR discuss how this contributes to the formation of memories and the the sense of presence in VR context. Ernst and Banks show that in any given perception task, the nervous system combines information from different sensory sources in a statistically optimal fashion [41]. They show that a maximum-likelihood estimation is a very close analogy to how the nervous system combines different sensory cues during a visual-haptic task. Although vision is often the dominating sense, the dominance under specific circumstances is determined by the lower variance associated with a sensory input. Ernst and Banks define noise thresholds for the maximum likelihood estimation and report haptic-capture-like behavior under experimental conditions where the visual noise crosses their empirically defined noise threshold. The experiments was conducted using height estimation as a measure. In a different paper, Ernst et al. show that haptic cues can have an effect on slant perception even if visual inputs are received after the haptic stimulation concludes [42]. In their experimental findings, the haptic feedback is adjusting the weights that the visual apparatus assigns to different cues during perception. They also report that the effect of the haptic cues on the slant perception can ’linger’ for an extended time period, retaining the weight shift. Lackner et al. explore the contributions of different body-perception sensory sources during the processes of spatial perception and orientation [43]. They provide an overview of the peculiarities of vestibular, proprioceptive and haptic inputs and their role in spatial orientation. Furthermore, expanding on these peculiarities, they describe sensory stimulation methodologies to elicit perceptual responses. Most relevant for this thesis is the consensus that proprioceptive responses can be achieved via vibrating specific muscle groups. Lackner et al. discuss the following collection of relevant work: • The illusion of head rotation and displacement can be elicited via vibration applied to the neck muscles [44]. • The illusion of the arm being extended further than it actually is can be achieved via vibration introduced to the biceps [45]. • Vibration applied to the Achilles tendon of a standing person can cause a feeling of forward pivoting around the ankle [46]. • Generalizing these notions, Lackner et al. show that that the illusion of displacement can be elicited in any body segment or the body as a whole [47]. Hagbarth and Eklund report the appropriate range of vibration frequency for activating muscles receptor endings to be around 100-120 Hz [48]. Todd et al. show that the vestibular system is also sensitive towards lower-frequency vibrations by measuring vestibular response directly instead of relying on participant reporting [49]. They detect vestibular responses at frequencies as low as 12.5 Hz. 11 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2. Related Work Typically, designers of a VR experience aim to stimulate only some of the sensory systems, leaving the rest to the characteristics of the real world workspace. However, research shows that in the context of VR, the simultaneous stimulation of multiple sensory inputs has an extensive application potential to shape user experience. Dinh et al. demonstrate that multi-sensory stimulation increases the user’s sense of presence, the feeling of immersion, and facilitates the forming of memories in virtual environments [19]. They evaluate the effects of visual fidelity, tactile, auditory and olfactory cues. They report that especially tactile and auditory stimulation can have a strong effect on the sense of presence and the forming of memory. At the same time, increasing the level of visual fidelity does not yield stronger responses, leading to the conclusion that the introduction of additional sensory cues is more important than increasing the fidelity of existing ones. Marchal et al. explore the effects of multi-sensory stimulation for walking experiences in virtual environments [50]. They provide a review of techniques that employ auditory, haptic and multi-modal rendering to simulate walking over different types of virtual ground. In addition to the vibrotactile and other haptic stimulation modalities dis- cussed above, they stress the positive effects of auditory stimulation on the sense of presence and immersion. Although non-conscious contributors, the auditory rendering of ground surface-dependent footstep sounds prove to be a high-impact, inexpensive tool for strengthening the sense of presence. Conflicting sensory information is a potential problem when designing multi-modal stimulation techniques. In the real world, conflicts in the sensory information from different sources can lead to sensory misalignment, especially during motion. This might cause to the manifestation of motion maladaptation syndrome [51]. In everyday use, this is referred to as motion sickness. In a VR context, this phenomenon is identified as simulator- or cybersickness, and is generally understood to be a detriment to the experience that has to be mitigated via careful interaction design [52]. Strategies to understand and counteract these effects have been widely studied [53]. We concentrate on works that propose mitigation techniques based on multi-sensory stimulation in order to augment or suppress conflicting sensory input. Weech et al. address this problem by proposing additional vestibular stimulation to mitigate simulator sickness symptoms [54]. Their method utilizes bone-conducted high frequency vibrations to stimulate the vestibular system. In a different publication, Weech et al. also propose the use of noisy galvanic stimulation of the vestibular system for the same effect [55]. Neither the bone-conducted vibrations nor the noisy galvanic stimulation are applied as a simulation/mimicking of the missing vestibular cues. Instead, both stimulation approaches are employed as a means to facilitate a sensory re-weighting. Both studies produce experimental evidence to the maximum likelihood estimation of sensory cues as described by Ernst and Banks [41]. The noise introduced to the vestibular system renders the signal ’unreliable’ and reduces the weight that the nervous system assigns to it. At the same time, the sensory system increases the weight assigned to the lower variance visual input, mitigating the effects of cybersickness. In contrast to the previous two approaches, Peng et al. [56] report a significant reduction in cybersickness via vibrotactile vestibular 12 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2.3. Multi-Sensory Stimulation for VR Figure 2.3: Illustration of vestibular stimulator prototype from Peng et al. [56]. They report the best results with the depicted 2-sided vibrator placement setup. stimulation that is synchronized to the steps of a simulated walking motion. They tested several different stimulation conditions (visual only, visual with head bobbing, auditory, tapping, synchronized/random vibrations with different vibrator placements) and report that 2-sided synchronized vibrations score lowest on average relative simulator sickness questionnaire and average discomfort, while scoring high on average realism. Refer to Figure 2.3 for an illustration of their prototype depicting ideal vibrator placement. A different direction in the research of multi-modal stimulation suggests that sensory conflict can also be used for the benefit of the VR experience. Marshall et al. provide a systematic overview of design strategies and suggest a typology of creative applications with purposefully introduced sensory misalignment [57]. They present the concept of sensory alignment as a design dimension and explore potentially applicable misalignment effects for different research areas (HCI, sensory science, non-digital entertainment). They create a framework for sensory alignment, in which design strategies are categorized around the level of alignment between physically and digitally stimulated senses. The three major categories are as follows: • alignment: for creating consistent simulation, • imperceptible misalignment: for expanding the possible range of effects on a system, • extreme misalignment: to create thrilling sensations. For the research presented in this thesis, the design strategies associated with the category of imperceptible misalignment are of highest importance. Marshall et al. assign this category to techniques that use very gradual alterations of perception, exploit and/or misdirect people’s shifting attention. Thereby, the limitations of human the sensory system can be leveraged to work around the limitations of simulation technology [57]. 13 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2. Related Work Our uneven surface simulation approach relies on imperceptible manipulation of visual cues. Through this approach, we aim to utilize the dominance of the visual sensory apparatus to elicit visual capture between the misaligned sensory cues, and use other sensory stimuli to mask the possible sensory conflict. 2.4 Locomotion and Redirection Real, natural walking can create highly convincing experiences, since it does not introduce another layer of abstraction between intention and simulated action. it has been show to have a positive influence on the following metrics: • Sense of Presence: Usoh et al. show that real walking causes a significant im- provement on the user’s subjective sense of presence in contrast to flying and walking-in-place [4]. • Spatial Updating: Chance et al. show that actual walking in a virtual maze is superior with regards to subsequent directional estimation of encountered objects, as opposed to joystick controls and a hybrid method [5]. They report a significantly lower degree of motion sickness associated with walking. • Search task performance: Ruddle and Lessels show that among the locomotion metaphors similar to the ones used by Chance et al. [5], real walking outperforms the other metaphors in search task performance as well [6]. When using real walking, users observe the layout of the virtual environments more closely, as evidenced by the fact that they tend to circumnavigate virtual obstacles that they could pass through. The authors infer that both the rotation and the translation of the body is necessary for accurate spatial updating. • Attention: Suma et al. report better task performance in divided-attention-type task when using natural walking [7]. Their experimental setup controls for both gender and task complexity. From the study results, they infer that natural walking consumes less cognitive resources that other, less natural locomotion metaphors. However, the use of real walking in VR applications is severely limited by the size constraints imposed by the available physical workspace. To ease this restriction, different techniques were proposed to remap virtual and physical space. The remapping of virtual and physical space enables to extend the size of virtual environment beyond what the physical workspace allows. Some techniques achieve this by manipulating the layout of the virtual environment. Suma et al. suggest the exploitation of change blindness by rearranging architectural features in the VE while the user is looking in a different direction [58]. In a different paper, Suma et al. suggest self-overlapping impossible spatial layouts to maximize the area that can be walked [59]. Vasylevska and Kaufmann propose a technique to 14 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2.4. Locomotion and Redirection dynamically generate self-overlapping layouts called "flexible spaces" via a procedural method [10]. In a different publication, Vasylevska and Kaufmann explore how different properties of the layout of self-overlapping impossible spatial arrangements influence human spatial perception [11]. Other techniques manipulate the users’ perception of their own location and/or orientation. These are referred to as redirected walking, as proposed by Razzaque et al. [9]. In this paper, the authors define their goals as retaining the high presence benefit of natural locomotion, while mitigating the limitations of confined tracking spaces. The underlying theory states that human spatial orientation and balance rely on visual and vestibular cues. During walking in the real world, vestibular stimuli caused by natural head rotations are compensated for by the visual system to counteract and maintain direction. The proposed technique relies on injecting additional rotation (rotational gain) in the virtual environment during natural head rotations to reorient users in the desired direction. If this additional rotation is of lower frequency than the natural head rotation, the user should not be able to notice it. Extensive research has been conducted to define thresholds for translational, rotational and curvature gains, exceeding which will make the redirection noticeable. Keeping gains below these thresholds also helps to mitigate the symptoms of cybersickness. Steinicke et al. [60, 61] provide a baseline for gain magnitudes for redirected walking. They discuss the different types of applicable gains. The human locomotion is described by a triple of three normalized 3D vectors (s,u,w), describing the strafe, up and walking directions respectively. These represent the basis for the gains to be applied to. • Translation gain is calculated by scaling the difference between current and previous tracked position by a factor derived from a mapping between tracked and virtual world distances. This allows for increasing or decreasing the distance travelled in physical space. • Rotation gain is defined component-wise on the three vectors of the human loco- motion triple as the quotient of the tracked and the virtual rotation angle. This type of gain can be used to suggest an orientation different from the actual tracked heading. In most cases, rotation gains are applied to the up vector (yaw). • Curvature gain can be applied to tracked movements by introducing small rotations as a function of the distance travelled. Curvature gains can be defined with the radius of a circular arc. This makes it possible to bend the walked path as the user compensates for the introduced offset. Although the thresholds determined by Steinicke et al. are widely used, developments in redirection methodology allow further refinement. Grechkin et al. revisit the the established thresholds [62]. They report that simultaneous application of translational and rotational gains can be regarded as safe. They also find that the arc radius used 15 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2. Related Work Figure 2.4: Table summarizing the taxonomy of redirection techniques, including some examples from Suma et al. [63] to calculate the curvature gain can be reduced depending on the employed redirection method. To create common grounds for discussion, Suma et al. suggest a taxonomy for redirection techniques and categorize previous work [63]. They propose three ordering concepts: • Gain type: Whether the redirection technique is concerned with the reorientation or repositioning of the user. • Noticeability: Whether gains are applied overtly or subtly. • Application time: Whether gains are applied discretely, or continously over time. Refer to Figure 2.4 for an overview of the taxonomy. Our experimental setup utilizes four different types of redirection techniques in two distinct application time groups to create an illusion of locomotion through a large virtual environment: • Continuous subtle repositioning/reorientation: Redirected walking with transla- tional and rotational gains below the established thresholds [60, 61, 62]. To integrate RDW into research and entertainment projects, Azmandian et al. developed a free toolkit for the Unity 3D game engine [64]. Our experimental setup utilizes this toolkit. Refer to Figure 2.5 for an illustration of the capabilities the toolkit offers. 16 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2.5. Vertical Locomotion and Uneven Surfaces Figure 2.5: Illustration of the built-in capabilities of the Redirected Walking Toolkit for Unity from Azmandian et al. [64]: Views and visualizations (left), inspector configuration parameters for the redirection (right). Configurable parameters include upper and lower bounds for translation and rotation gains, radius for calculating curvature gain and custom head transform object reference. • Discrete subtle repositioning / reorientation: Sudden introduction of gains without the user noticing. Suma et al. [63] reference some techniques of these types, but also underline the rarity of such techniques due to the inherent difficulty of applying discrete gains in a subtle manner. Bruder et al. exploited optic flow effects on peripheral vision and change-blindness via inter-stimulus images to hide the discrete application of gains [65]. They covered the periphery of the user with visual effects for a few frames and used these different effects to simulate self-motion. For an overview of all the effect they experimented with, refer to Figure 2.6. We employ a technique comprised of a combination of optic flow and fixated attention induced change blindness. 2.5 Vertical Locomotion and Uneven Surfaces Physical workspaces used during VR experiences are even more limited vertically than along the horizontal dimensions. Due to health and safety risk, the physical area is usually completely flat and level. In contrast, real world environments are rarely perfectly even. Therefore, diverging the locomotion in VR from a purely horizontal surface is an important step towards more realistic virtual environments. Magical metaphors, such as teleportation [66] and flying [4] are the easiest to implement vertical locomotion. These metaphors offer a greater freedom of movement in VEs, but they are less natural and very different from real world experience. Considering the limited workspace, Slater et al. suggest to use virtual stairs or ladders in combination with the "walking-in-place" metaphor for vertical locomotion [67]. Lai at al. developed the ladder climbing idea further, adding hand and feet tracking as well as grab gestures using handheld controllers [68]. Matsumoto at al. describe means of evaluating the amount of energy spent walking on a upwards or downwards slanted surface as opposed to waking on level ground. They 17 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2. Related Work Figure 2.6: Overview of all the effects tested by Bruder et al. [65]. They propose different visual effects for creating a self-motion illusion: "(a) particles, (b) sinus gratings, (c) and textures fitted to the scene, respectively (d) contour filtering, (e) change blindness and (f) contrast inversion. Illusory motion stimuli are limited to peripheral regions." employ translation gains to replicate the increased energy consumption needed for walking uphill [69]. They simulate vertical locomotion through real horizontal walking, with gain controlling the distance as a function of hill steepness. Marchal et al. evaluate the effectiveness of different camera manipulation techniques for simulating height changes in virtual environments [13]. The research is motivated by the limited usefulness of large, heavy and expensive physical terrain rendering apparatus. Instead, the proposed interaction techniques borrow from desktop-based virtual environ- ments, such as first person shooter games. They propose four different types of camera manipulations. A comparative analysis of the effectiveness of these visual feedback-only methods is presented based on a small-scale user study. The evaluation is interested in the ability of the users to detect and correctly recognize holes and bumps on the ground of the virtual environment. The techniques proposed by Marchal et al. manipulate different view-port properties as 18 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2.5. Vertical Locomotion and Uneven Surfaces Figure 2.7: Illustration of the camera manipulations applied by Marchal et al. to simulate walking on an uneven surface [13]. a function of height in the virtual environment: • Changing the height of the camera: The vertical position of the viewport is increased/decreased based on the relative position of the user to the virtual un- evenness. • Changing the orientation of the camera on the pitch axis: The viewport is tilted upward or downward while traversing from the edge to the center of the virtual hole/bump. • Modifying the camera movement velocity: The traversal speed is slowed down while moving up and sped up while moving down. • The fourth technique is a simultaneous application of all three of the above. All four techniques utilize a Gaussian profile to allow a smooth transition from flat to uneven virtual ground. For an illustration of the camera manipulations employed, refer to Figure 2.7. It is important to note that Marchal et al. did not provide a visual representation of the virtual unevenness. They only mark the area on the ground in which the manipulations take place, but don’t visualize whether the unevenness is a protrusion or indentation. Their between-subject user study evaluates the user’s ability to recognize the type of virtual unevenness (bump, hole or none) and subjective user experience. The movement 19 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2. Related Work direction (walking forward or backward) and the application environment (HMD or desktop setup) are varied. Subjective user experience is measured through four metrics: Ease of judgement, realism, cybersickness and global appreciation. In addition, the participants had to guess the height/depth of the virtual unevenness. The authors conclude that there is a significant correlation between the effect applied and the user experience. For our purpose only the HMD case is relevant, discussion of the desktop results are omitted. The orientation method enables easiest recognition. It also causes the highest estimation of height/depth, an overestimation in all cases. At the same time, a high number of users finds this effect to be detrimental to immersion. The velocity manipulation is the least well received and the least effective. The combination of different techniques yields the most positive global appreciation. The uneven surface simulation approach discussed in our thesis is partially building on these findings from Marchal et al. [13]. 2.5.1 Non-Flat Virtual Environments and Multi-Sensory Stimulation Uneven surfaces with corresponding haptic sensations are intrinsic parts of the real world and its multi-sensory representation. Unfortunately, both are often lacking in VR. Physical objects and uneven surfaces in the VR workspace pose a potential danger to the user if their registration or representation in the virtual environment is not 100% accurate. To find a solution for a low cost, safe stair simulation, Nagao et al. propose a passive haptic feedback method [14, 15, 16]. The feedback is provided by angled bars placed equidistant from each other on the level ground of the tracking space. These bars represent the edges of the stairs. The user’s shoes are tracked and represented in the virtual environment. The authors report results of a small-scale user study and conclude that the haptic feedback and the visual representation of the users feet in combination provides a heightened sense of immersion. Refer to Figure 2.8 for an illustration of the idea. Asjad et al. investigate the perception aspect of this technique further by varying different factors, such as visual cues (feet tracking), haptic feedback, walking direction (ascend/descend) and introducing interference factors (fear of height and fear of closed spaces) by changing the environment from open stairs to a closed stairwell [70]. They report that height was generally overestimated by participants, and conclude that the passive haptic feedback does not make a significant difference. A number of techniques rely on electro-mechanical devices to create a physically changing environment. Iwata et al. propose a device with platforms that trace the user’s feet and return them to the neutral position [17]. The platforms move horizontally to allow walking on flat ground and can be raised vertically to simulate stairs. Yoon et al. proposed a similar device consisting of two 6 DOF parallel manipulators to simulate walking on different uneven virtual terrains [18]. Their device is capable of sensing human walking motion and adapts the actuations accordingly. They evaluate different virtual terrains, such as level ground, sloped surfaces and stairs and report that users were able walk 20 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2.5. Vertical Locomotion and Uneven Surfaces Figure 2.8: Illustration and demonstration of infinite virtual stairs via passive haptics from Nagao et al. [14]. Angled bars are placed on the floor equally spaced. The haptic sensation of stepping on the bars combined with manipulating the height of the virtual camera creates an illusion of ascending/descending stairs. Figure 2.9: Electro-mechanical solutions for uneven surface simulation in VR. Gait Master from Iwata et al. [17] (left) and 6 DOF parallel manipulators from Yoon and Ryu [18] (right). naturally, start and stop safely and change walking speed as desired. For an illustration of these electro-mechanical devices, refer to Figure 2.9. Other solutions incorporate whole-body vibrotactile stimulation. Vasylevska and Kauf- mann propose the elevator metaphor implemented with a multi-sensory stimulation platform that vibrates in accordance with the audio recording of a real elevator [71]. Figure 2.10 illustrates the VE and the device used in their user study. They compare this metaphor to flying and teleportation and show that the elevator is more natural, increases the sense of presence and better supports the spatial orientation than other methods. This work inspired us to introduce the vibrotactile stimulation as a whole-body 21 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 2. Related Work Figure 2.10: Virtual environment and vibrotactile elevator simulation device from Va- sylevska and Kaufmann [71]. vibration in our project as well. It is a less intrusive method of delivering the stimulation than head-mounted vibrators employed by other approaches. The diversity of research topics discussed in this chapter highlight the wide array of considerations that have to be acknowledged when designing a multi-sensory stimulation approach. In the following chapter, we discuss the conceptual design of our approach derived from the insights gained from the review of previous work. 22 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . CHAPTER 3 Conceptual Design The goal of our thesis project is to implement and evaluate such a VR setup for uneven surface simulation. In practical terms, we aim to create a simulation of multiple bridges in VR that can be parametrized to different visual heights, while using only one physical bridge. We chose to use bridges for the following reasons: • The bridge is a real-world object that can be integrated into VR. VEs can be designed in a way where the bridge is an organic and familiar part of the world and the interaction. • Choosing the appropriate type of bridges allows both ascending and descending. Using moon bridges and hanging bridges allows for walking both up and down, while also ending up on the same horizontal plane after crossing. This enables inserting the bridge seamlessly into the otherwise flat physical workspace. • Physical bridge props can be specifically designed and constructed to provide the safety measures required for low-risk user testing. They enable controlled usage by design, since they provide familiar interactions, such as stepping up/down and grabbing the railings for support if necessary. Based on the review of the related work, we selected the following four factors to investigate: 1. Shape of the physical bridge prop: we use two different physical bridge props. One has an actual upward curving, uneven surface. This bridge prop allows real vertical traversal, i.e. physically ascending and descending. The other one is flat, even, but still provides the haptic feedback associated with being on an actual bridge. 23 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 3. Conceptual Design 2. Visual height of the virtual bridge: We vary the visuals of the virtual bridge model. We create both upward curved bridges and hanging bridges. Coupled to the visual height of the virtual bridges, we manipulate the height of the virtual camera viewport. By raising the camera’s virtual height and increasing pitch rotation according to the characteristics of the simulated surface, we convey the feeling of ascending or descending on a slanted surface. 3. Camera pitch manipulation: We apply pitch rotation to the virtual camera viewport according to the curve of the virtual bridge, i.e. the slant of the virtual surface the user is standing on. 4. Vibrotactile stimulation: We introduce a whole-body vibrotactile stimulation to the user via the entire surface of the bridge to introduce noise into the vestibular system. We utilize this noise to render the vestibular and proprioceptive cues less reliable for height and slant perception, thus increase the effect of the simulation. The multi-sensory stimulation is implemented as a combination of different methods found in the related literature, including visual stimulation via camera viewport manipulation, passive haptics and active vibrotactile stimulation. Based of these factors, we formulate the following three research hypotheses: • H1. An uneven physical surface will allow for larger height manipulations. • H2. The added pitch will increase the perceived slant, especially with an uneven physical surface. • H3. Added vibrotactile stimulation will increase the perceived height and slant, and even more so for an uneven physical surface. Empirical user studies allow us to evaluate the influence of the chosen factors on the success of the uneven surface simulation. We evaluate the factor of the physical bridge prop shape and test out main hypothesis H1, by performing two independent user studies, each with a different physical bridge. To evaluate the other factors and test hypotheses H2 and H3, we chose a within-participants experimental design for both of the two independent user studies. In both studies, we test virtual bridges with several visual heights and all possible combinations with the other binary independent variables (present or not) . The sequences of bridges were generated using a balanced Latin square. Additionally, each bridge sequence was counter-balanced to compensate for the potential learning effect. 24 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . CHAPTER 4 Implementation In this chapter we provide details about the tools that we created to test our hypotheses. In the first section, we describe the physical devices (bridge props) we built in order to introduce haptic and vibrotactile stimulation. In the rest of the chapter, we describe different parts of the software application that we developed in the course of the presented research. The discussion is partitioned along main design concepts and goals. We present the problems we encountered during implementation and describe their solutions as used in the final software application. 4.1 Physical Devices To test our hypotheses, we have constructed two physical footbridge props. Out hypothesis H1 assumes that using the passive haptic feedback of an actual physical uneven surface allows for larger height manipulations. To test this, we have created the two props with similar features, except for one important difference: one of the props is a flat bridge, while the other one is an upward curved bridge. Apart from this one feature, from the perspective of the user, the two bridges are functionally identical: they have identical width, length, railing height, haptic feedback of materials, etc. Therefore, we can isolate the condition of the actual physical uneven surface when conducting user studies. The main building materials for both props is lumber. Additionally, both have some metal and plastic parts. Both bridges have the following key features: • A structured walkway with orthogonal planks. • Railings for support, safety and to help users with orientation when approaching the bridge. • A supporting metal structure with a Buttkicker LFE audio transducer affixed to the bottom of the prop. The metal structure provides additional rigidity and a 25 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation strong attachment point for the audio transducer. The metal helps to propagate the vibrations more evenly throughout the entire length of the bridge. • Railings made from 60mm smooth plastic wall-mountable cable channels, supported by wooden posts attached to the bridge. The cable tunnels provide a sturdy, uniform and smooth gripping surface that is safe to touch and grab. Figure 4.1: Illustration of the flat bridge prop. Top: CAD drawing of the finalized design, used to evaluate stability and cost efficiency; Bottom: Finished construction of the flat bridge prop. As mentioned above, the two props differ mainly in their shape. The distinct characteris- tics are as follows: • Flat bridge: no height difference in the walkway. Refer to Figure 4.1 for an illustration of the design and finalized construction of the flat bridge prop. For the wooden parts we used pine and particle board. The walkway is made of a strong 22mm particle board base. Thinner 12mm particle board stripes attached to the top simulate the structure of orthogonal boards for passive haptic feedback. The 26 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.1. Physical Devices audio transducer is attached via a metal arm to a thick supporting aluminium rail on the bottom side of the walkway. The arm is made from 5mm steel sheet metal. The flat bridge is elevated 120 mm from the ground, resting on square pine girders, allowing for the aluminum rail to fit on the underside of the walkway. A channel in the middle allows for appropriate clearance. Figure 4.2 shows the mounting solution of the audio transducer. The plastic railings are held up by soft lumber beams, aiding orientation and allowing for the occasional regaining of balance. Figure 4.2: Left: Audio transducer mount on the flat bridge prop: Underside of the walkway with aluminum rail; Right: audio transducer mounted on metal arm. Figure 4.3: Flat bridge disassembled into components for transport and storage. • Curved bridge: Curved surface, with 300 mm height difference between the first step-up and the the apex. Refer to Figure 4.4 for an illustration. The bridge rests on a welded steel arch made from sheet metal stripes (5mm thickness). The arching shape of the stripes is kept up and supported by steel rods cut to appropriate lengths from 10mm diameter round barstock. The rods have sheet metal feet for an increased stability, and to protect the ground beneath from tears and scratches. Since the arch allows for appropriate clearance, the audio transducer is bolted directly on the underside of the walkway. Refer to Figure 4.5 for an illustration of the metal structure and the audio transducer mount. The orthogonal planks are bolted to this metal arch with round-head carriage bolts to reduce tripping risk. We used lumber of high density and rigidity to minimize swaying and deformation. Heavy wooden girders support the plastic railings, which allow for a more stable support. This is important because of safety considerations regarding actual vertical traversal. As ascending can be more demanding on users, 27 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation making them more likely to lean onto the railings for support. The walkway of the bridge has a step-up of 65mm. During the early planning phase of the project, we have developed an additional physical bridge prototype that can be set to different physical peak heights. Refer to Appendix A for a detailed description. Although cost was a prohibitive factor for the current research, future work can potentially benefit from an adjustable physical prop. The final design of the physical bridge props was motivated by the goal of creating a relatively low cost device that is reasonably easy to replicate. The resulting props are sturdy and provide appropriate safety to allow low-risk user testing. The props can be disassembled into smaller modules for easy transportation and storage. Figure 4.3 shows the flat bridge prop disassembled into components. Figure 4.4: Illustration of the finished curved bridge prop. Figure 4.5: Supporting welded steel arch structure and audio transducer mount on the underside of the curved bridge prop. 28 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.2. Software 4.2 Software To test our hypotheses, we have created a complex application. The following section contains descriptions of crucial parts of this application, detailing their design-, function- and task-specific aspects. The application was created as a projects via the Unity game engine [72]. We chose the Unity game engine because there are different extensions and frameworks available on the platform. These provide either pre-existing functionality, or a good basis for for developing custom functionality needed for our research: • Virtual Reality Setup: The Unity game engine has a well documented, constantly updated SteamVR [73] integration. The integration is provided via a plugin [74] that can be easily imported into a project. This plugin allows access to tracking data from SteamVR tracked devices (HMD, controllers and tracking pucks) and to the HMDs stereo rendering. • Redirected Walking: Azmandian et al. [64] created the Redirected Walking Toolkit (RDWT), made available for research projects. The RDWT provides a good basis for our redirection requirements, lessening the implementation overhead. • Rapid Prototyping: The Unity game engine allows for a very fast creation of custom virtual environments. Standard and extended assets are available free of charge and can be utilized to create purpose-built virtual environments without time-consuming modeling and texturing work. 4.2.1 Virtual Environment and Virtual Bridges The virtual environment we have created for the user studies consist of a mountainside with a small lake at its center. On this lake, artificial islands are connected via bridges. Each island features an aforementioned reporting station and provides a stage for the balloon game used for redirection. See Figure 4.6 for a birds-eye overview of the VE. Islands are set up in a modular way to allow any study configuration, with an arbitrary number of bridges present in the VE. We have implemented the Module Spawner component script to automatically set up a scene according to the requirements of a specific test scenario. We have designed the virtual bridge model to allow any desired bridge shape. The bridge model is set up as a hierarchical collection of parts, which can be moved and rotated by the Bridge Constructor script. This process dynamically creates the desired visuals for each bridge of a test scenario, according to pre-set test conditions. For the two different physical bridges and the corresponding different height and slant adjustment methods, we have implemented two distinct bridge construction methods: • To use with the flat bridge prop, we sample sine wave in the [0, π] interval with the amplitude scaled to the desired bridge height and the interval scaled to the desired bridge length. The sampling is done at the location of each walkway plank object along the centerline. The resulting height is assigned to the corresponding plank. 29 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation Figure 4.6: Birds-eye view of the VE we created for our user studies. The virtual bridges depicted are of the type we used for the flat physical prop in our user study 1. Top: Overview of the entire VE with the surrounding mountainside and lake; Bottom: Close-up partial view of the VE with islands connected via differently configured virtual bridges, arranged in a consecutive crossing pattern. The tangent line orientation is calculated at each plank location as well, and the corresponding plank is rotated on the horizontal axis perpendicular to the walkway (i.e. bridge X axis) to match this orientation. This yields the visuals of a smooth, properly constructed curved bridge / hanging bridge walkway surface assembled from individual planks. Both the walkway support railing and the handrails are comprised of identically placed small elements, and are positioned and oriented via the same process. • The curved prop posed additional challenges, since the physical prop was neither completely symmetrical, nor uniformly curved. To prevent a noticeable disconnect between the virtual and the physical prop (especially in the cases of extreme added height values), we created a different algorithm for the virtual bridge model setup. We measured the actual height of the prop at the position of each physical walkway plank, and used the measured values to define a curve representation of the bridge walkway profile. We repurposed the Unity animation curve component to store this curve, and used it to sample the virtual walkway plank positions. Plank orientations have been calculated numerically. Since the railings of the curved prop 30 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.2. Software had a different profile than the walkway, we used a second animation curve to store it’s sampled values and construct the model geometry. Refer to Figure 4.7 for a visual comparison of the physical prop, the sampled curves and the parametrized virtual bridge model. Figure 4.7: Construction of virtual curved bridges. From top to bottom: Real curved prop for reference. Observe the artifacts in the walkway and the railing profile created by our limited resources during construction; Sampled walkway profile stored as Unity animation curve; Sampled railing profile stored as repurposed Unity animation curve. Both curves are depicted not to scale, but stretched in height to illustrate that they are the result of sampling points on the physical bridge walkway and railing for their height.; Virtual bridge created by using the repurposed animation curves. We are sampling them at the location each element or the virtual bridge model to create proportionate visual representations in the VE. As described above, test conditions can be defined in batch files and automatically loaded by entering the desired test case number into the Test Case Manager component. Each bridge in the virtual environment is set up automatically to have the right visuals and height/slant adjustment values for the given test-case. This prevents mistakes during user study sessions and allows quick resets for consecutive participants. 31 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation 4.2.2 Multi-sensory Stimulation We use multi-sensory stimulation, comprised of the following components: • Passive haptics: As described in the previous section, the curved physical bridge prop provides the proprioceptive cues of an uneven surface. The physical bridges provide the passive haptic feedback of actually stepping onto/off from a real bridge, feeling the structure of the planks, and touching the railings. • Camera manipulations: Change of height and pitch rotation of the virtual camera viewport to simulate visual cues of walking on an uneven surface. • Vibrotactile stimuli: Whole-body vibrations introduced through the bridge via an audio transducer. Visual Manipulations Our height and pitch manipulations are inspired by previous work by Marchal et al. [13], in which they describe camera viewport manipulations. However, our manipulations are strongly coupled to the visuals of the simulated uneven surface, while their approach employed an abstract bump or hole simulations without visual distinction between them. Marchal et al. introduced three different types of camera manipulations: height adjustment, pitch rotation and velocity manipulation. Our approach omits velocity manipulation due to it’s poor performance as reported by Marchal et al. Refer Section 2.5 for further details. Another reason for this omission is that one of our physical props, the curved bridge, has an actual uneven surface, with a real geometry, i.e. a fixed apex location along the length of the bridge. The camera velocity manipulation technique is not compatible with such an arrangement, as it requires to shift the apex of the simulated height. With an physically uneven surface, the actual and the simulated apices would not meet correctly. Camera Height Manipulation Marchal at al. [13] use the position of the HMD for their calculations. We found that this does not work correctly in our planned test scenarios, where users can move and look around freely at their own discretion. Leaning forward/backward or generally interacting with the VE on the bridge causes the manipulations to become unnatural and obvious. An alternative means of determining user position is tracking the user’s feet. This solution carries the added benefit of heightening the sense of presence through increased body ownership, as shown in a similar context by Asjad et al. [70]. Refer to Section 2.5.1 for specifics. However, due to limitations introduced by the physical bridge props, we were unable to use the feet positions. The high degree of occlusion created by the railings made the feet tracking too unreliable. At the same time, the railings were indispensable due to safety concerns. Therefore, we resorted to placing a tracker on the lower back of the user, at the "tailbone" position. Refer to Figure 4.8 for an illustration of tracker positioning. This positioning is more stable than the head and more reliable than the 32 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.2. Software calculations based on the feet positions. From the tailbone position data, the location and orientation of the whole body can be inferred within a reasonable degree of certainty. During preliminary testing, we found this setup to yield adequate camera manipulations and to allow the user to move in the VE and on the bridges without restrictions. Figure 4.8: Position of the tailbone tracker used to calculate location of the user in the VE. This location data is used to determine the user’s position on the bridge and calculate camera manipulations. The height adjustment is calculated from a curve that describes the shape and is scaled to the dimensions of the virtual bridge. For further details about the the definition of the curve refer to Section 4.2.1. The curve is evaluated at the user’s position along the center-line of the bridge. For the height adjustment, we take the height value of the virtual bridge at the corresponding point and shift the camera viewport along the Y axis to correspond to that calculated height. In the case of the flat prop, this means simply shifting the camera to that exact height. For the curved prop, this means shifting the camera by the difference between the height on the calculated virtual bridge and the real height of the physical prop at the corresponding bridge location. Figure 4.9 illustrates this process. Camera Pitch Rotation Manipulation Similarly to the height adjustment, the pitch rotation adjustment is calculated from the user’s position along the centerline of the bridge. To get the slant of the bridge surface, a tangent line is calculated for the curve at the corresponding position. We use the angle between the actual slant of the physical prop and the calculated slant of the virtual bridge. The camera viewport if rotated by this difference to convey a slant of the surface similar to that of an actual bridge of corresponding height. The rotation happens around 33 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation an axis perpendicular to the bridge centerline (i.e. bridge X axis), going through the HMDs location. Figure 4.9: Illustration of camera manipulations used. However, preliminary tests showed that this amount of pitch rotation has to be scaled back in order to prevent disorientation, the breaking of the illusion or even dizziness. Our assessment is that when standing on a real physical slanted surface, the body and head pose is naturally adjusted towards a level direction. The body compensates for the slant at the ankles, hips and neck. Without appropriate proprioceptive feedback, the virtual slant introduced to the head (i.e. the camera viewport pitch rotation) alone is too intense. Via empirical testing, we decided on scaling the added pitch rotation angle to 50% of the calculated value. We also found that the pitch rotation manipulation was more noticeable when standing perpendicular to the bridge centerline. To compensate for this, we introduced another dynamic scaling factor that is dependent on the angle between the bridge centerline and the flattened camera viewing forward direction (i.e. HMD Z axis). The more the flattened camera viewing direction approaches a perpendicular angle to the bridge centerline, the lower the percentage of the calculated rotation that is introduced to the HMD. Additionally, we defined short transition areas at the two ends of the bridge, where the adjustment was gradually scaled down towards the ends. This is done to prevent a noticeable change of camera position and orientation when getting onto/off from the physical bridge. Controller Position Manipulation During the user study, two handheld Vive controllers are provided to the users. This allows them to input data about their experience. However, since the geometry of the virtual bridge differs from the shape of the physical props, touching the railings with the controllers would result in the controllers clipping through the virtual railings. This would break the illusion and endanger the users if they need additional support, but can’t locate railings. Therefore, we added height adjustment to the controllers as well. Each controller’s position is used as input for evaluating the curve associated with the bridge, and controller location is shifted along the Y axis by the calculated height value. 34 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.2. Software As a results, the controllers’ visual and passive haptic cues are in constant agreement. Controller models in the VR view are always contacting the virtual railings at the same time when the real controllers touch the physical railing. Even sliding the controller along the flat physical railing results in the virtual controller following the curve of the virtual railing accurately. Research on embodiment has shown that with congruent multi-modal stimulation, the location where the stimulus was observed can override the visual perspective of self-location. Tool embodiment is governed by similar principles, which is backed up by our experience as well: even though the height adjustment of the controllers dynamically shifts the perimeter of peripersonal space, very few study participants reported realizing this. For further information on embodiment, refer to Section 2.2. Vibrotactile Stimulation Our multi-sensory stimulation approach incorporates whole-body vibrotactile stimulation. We seek to test our H3 hypothesis, stating that the introduction of vibrotactile stimulation will increase the threshold of the height simulation. For an overview of the underlying theoretical background, refer to Section 2.3. The vibration is introduced via the entire physical bridge through the primary contact points of the user’s soles on the walkway and the secondary contact points of the user’s hands on the railing. To create the vibrotactile stimuli, we use a Buttkicker LFE audio transducer. Audio transducers work similarly to loudspeakers, generating vibrations via a permanent magnet and an electromagnetic coil, driven by an amplified signal. However, instead of vibrating a membrane to produce audible sound, the transducer vibrates the structure directly, to which it is attached to. Audio transducers are used for enhancing entertainment experiences by adding another sensory stimulation modality. The transducer was mounted onto the underside of the bridges attached to supporting metal structures. As an input signal, we use an audio file containing a uniform sinusoidal sound wave. We explored frequencies in the range of 20-60 Hz. Based on our preliminary tests, a frequency of 40 Hz was chosen to be the most effective. It was not as disturbing as lower frequencies (20 Hz - 30 Hz). At the same time, it could still be perceived as whole-body vibrations, unlike higher frequencies. Vibrotactile stimulation usually uses higher frequencies, around 100 Hz, as this frequency elicits the highest response from the vestibular system. Refer to Section 2.3 for further details. However, techniques proposed in relevant literature employ bone-conducted vibrations, introduced via vibrators placed directly on the user’s head. Out preliminary observations might be dependent on a number of factors, such as the contact points, or the dampening effect introduced by the construction materials and the vinyl flooring found in the lab workspace. We aimed at a slightly noticeable, but not disturbing or destabilizing effect. The amount of the signal amplification was empirically adjusted for the sensations to be comparable between the two physical bridges, since the two props differ in mass and shape. The transducer is turned on before user steps on a bridge and turned off after the user 35 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation left the bridge. We implemented the triggers in a way as to allow enough distance to the bridge when the vibration is activated/deactivated. Additionally, we use volume fade in/out to avoid the detection of any sudden start/stop of the transducer by the user. The vibrations are triggered by the user’s body position (tailbone tracker) entering the designated zone. The start/stop commands are relayed via a wireless network connection from the application running on the main desktop PC to a laptop connected to the transducer’s amplifier. 4.2.3 Redirection Redirection techniques had to be implemented in order to facilitate using a single physical prop to test a multitude of conditions. Our laboratory workspace spanned an area of 6 x 10 meters, with a single physical bridge prop in the center. On each side of the bridge, an area of 6 x 2 meters was available for achieving the redirection onto the next virtual bridge. We based our redirection implementation on the Redirected Walking Toolkit (RDWT) provided by Azmandian et al. [64] for the Unity game engine. Preliminary test showed that introducing translational and rotational gains up to the thresholds determined by Steinicke et al. [61] allows for redirecting participants onto the same single physical bridge if the virtual environment is set up properly. For further details on VE setup refer to subsection 4.2.1. Using the built-in redirection methods provided with the RDWT, we measured typical redirection times of around 90 seconds. We found these results to be too slow for our purposes, as crossing 16 bridges would include a net redirection overhead of 24 minutes. This has the potential to cause boredom and ultimately fatigue among study participant and would prohibit us from testing the number of variable combinations that we planned for the user study. Additional challenges arose from the need to achieve both near-to-exact positioning and orientation (i.e. registration) of the virtual bridge model to the physical prop. This was a hard requirement for the redirection, because even 5-10 centimeters or 1-2 degrees of separation between the virtual and the physical bridge would lead to a broken illusion, if users continue walking on a physical bridge that is misaligned with the virtual model. Thus, redirection had to be implemented in a manner that is both fast and precise To combat the time and physical space constrains, we devised a novel redirection technique based on a combination of change blindness and optical flow simulation. According to the taxonomy of redirection techniques in immersive virtual environments as outlined by Suma et al. [63], our solution belongs in the category of subtle discreet repositioning/reorientation. The technique is deployed as a game of popping balloons. In this game, the user gets tasked with popping balloons by walking up to them an touching them with the VR controller. For additional information, refer to Section 5.1. While the redirection is active, colorful balloons are spawned around the user. The spawn location of a balloon is determined by the following factors: 36 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.2. Software • Optimal redirection: When popping a balloon, the optimal (maximal) repositioning and reorientation is calculated from the known location and orientation of the physical bridge prop in the workspace and the next virtual bridge model in the VE. Furthermore, if the angle between the real and the virtual bridge has not been closed yet, balloons are spawned behind the user to motivate turning. • Spatial constraints: Both the constraints of the physical workspace and the VE have to be observed to avoid directing the user into dangerous locations. Observing these factors, the next balloon is spawned at the intersection of the flattened optimal repositioning/reorientation direction vector originating from the user’s current location and the boundary of the physical workspace or the current VE location, dependent on which boundary is closer. Figure 4.10 illustrates this process. Figure 4.10: Illustration of how balloon spawn points are determined by observing optimal redirection potential and physical / virtual spatial constraints. When the user fixates his/her view on the balloon to touch it, the balloon’s position relative to the virtual camera is stored. On touch, the balloon disappears and the field of view is filled by a dense cloud of colorful spherical particles, simulating confetti. At the same time, extra rotation (≤5°) and translation (≤30 cm) are added to the player pose to reduce the angle and distance between the physical prop and the next virtual bridge. To keep the manipulation unperceived, the particle emission center is moved together with the camera, maintaining the previously-stored relative position. Figure 4.11 demonstrates this process. Preliminary tests showed that mean redirection time could be reduced significantly, from over 90 seconds (RDWT built-in steer-to-redirection) to less than 20 seconds (our 37 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation Figure 4.11: Demonstration of redirection introduced during balloon pop. Left: Balloon intact; Center: balloon popped, confetti effect; Right: the two frames overlaid to visualize the rotation introduced at the moment when a user touches a balloon. technique combined with the built-in steer-to-redirection), depending on users’ behavior. Additionally, the final balloon pop could be used to mask the remaining reposition- ing/reorientation necessary to exactly match the pose of the next virtual bridge to the single physical prop. Very few study participants reported noticing being reposi- tioned/reoriented, leading to a conclusion that this technique merits further exploration in future work. 4.2.4 User Interface The application features two distinct sets of user interfaces: one for experimenters and one for users. Experimenters’ Interface and Control Options The interface for the experimenters is mostly comprised Unity inspector component panels on specific Unity Game Objects. We adapted this approach to maximize time efficiency while creating functionality for ourselves to use. In the following list we restrict ourselves to outline only the most important component scripts that provide functionality for experimenters. • Test Case Manager : provides functionality for setting up and conducting user study sessions. It facilitates loading test cases from a text file. It contains output to quickly double-check the correctness of test conditions. The Test Case Manager can be used to apply test condition parameters to appropriate game objects in the scene. This includes configuring virtual bridge models, camera manipulation and vibrotactile stimulation parameters. The Test Case Manager allows setting up vibrotactile trigger network properties. • Bridge Constructor : is a configuration interface for bridge model visuals. It can be used on a per-bridge basis or set up via the Test Case Manager. 38 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.2. Software • VR Height Adjuster : is a configuration interface for the multi-sensory stimulation settings. It can be used on a per-bridge basis or set up via the Test Case Manager. • Apply Prop Registration: allows for storing and applying bridge prop registration data. It stores raw tracking pose of registered points in a text file. It can be used to quickly reapply bridge registration, i.e. move and rotate the VE to match a particular virtual bridge in the scene to the one physical prop. • Module Spawner : facilitates creating uniform test scenes with a custom number of islands. • Vibration Tester : allows for double-checking the function of the vibrotactile network trigger before test sessions. It is also used to prepare the transformers of the transducer’s amplifier. • Vive Tracker Identifier : enables hard-coding Vive Tracker device information. It can be used to predictably assign tracking device roles via hardware id, even if devices are turned of/replaced. • Custom Logger : provides data logging management. For further details about our logging system, refer to Subsection 4.2.5. • Redirection Manager / Bridge Redirector Close Angle: Configuration and debug output for the redirection. Additionally, an overlay is added to the desktop display during the study session. The overlay displays information about the current test conditions and provides highlighted data about which bridge the user is currently crossing and which one is the next to follow. VR User Interface In contrast to the rapid design of the experimenter interfaces, we invested much more time when creating the user interfaces and interactions inside the virtual environment. Since the user study was planned to be conducted on the general public, the interfaces and interactions had to be unambiguous, easy to understand and to use. At the same time, the user input had to be in clear relation to the data that was intended to be captured. Thus, the design and implementation of these interfaces and interactions has been subjected to preliminary evaluations during the implementation phase. The interactive elements of the VE are color-coded. Clickable elements are colored red when available, or gray when inactive. Elements than can be grabbed and moved are colored green. For the grab-and-move action, we implemented the laser pointer metaphor, which is widely used in VR applications. When the user points at an interactive element with a handheld VR controller, a green line is rendered from the controller to the element, signaling the possibility to interact. At this point, pressing the trigger button on the controller enables to grab the element with the laser. The element can be moved on a restricted path (e.g. up/down for vertical sliders) until the trigger button is released. The laser line changes to a red color when an element is grabbed. 39 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation All user interface elements are set up to prohibit accidental or unintended input. Confir- mation buttons only become available (unlocked) once users provided actual input on the corresponding grab-and-move input element. Zero values can only be input intentionally, by moving a slider/pad from it starting position, then putting it back again. All user interface elements enable resetting to initial state (i.e. zero value) by a press of the controller thumb button. Input always has to be confirmed by the user by pointing and clicking on a confirm button. At the users confirmation, input is encoded and stored in log files. Logged data is immediately flushed to disk to prevent data loss in the case of a malfunction. During the user study, participants have to interact with three interfaces: • Height reporting arm: A panel affixed to the virtual bridge on a rotating arm. It contains a horizontal slider controlling a numeric representation of the height value in centimeters. Participants have to use this element to report the height of the bridge through a numeric value, while standing on the bridge. This slider provides a fine-adjustment input action that allows adjusting the numerical value to +/-1 cm by pressing the right/left side of the touchpad. The arm rotates in from a neutral position beside the bridge to face the users when they cross a trigger on the bridge, thus blocking the way forward. After providing and confirming an input, the arm rotates out of the way, signaling the go-ahead to continue. Figure 4.12 shows the reporting arm at different stages. Figure 4.12: Reporting arm on the bridge, used to inquire a numeric height value. Blocks bridge until input is confirmed. Left: Depicted as used on the flat bridge, with zero value in center, -/+50 cm extreme values; Right: Depicted as used on the curved bridge with a scale between 0 and 100 cm, depicted while manipulated by user with laser. • Reporting station - Phase 1 : A station after each bridge, on the following island. Phase 1 of the reporting station contains a vertical height slider and a tilting slant pad. The vertical height slider is marked at the zero position and at the 40 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.2. Software Figure 4.13: Comparison of experimental apparatus used to capture participant feedback about perceived slant by Proffitt et al. 1995 [21] (left), Proffitt et al. 2001 [22] (center) and the one we created for our user studies (right). maximum/minimum positions. The markings are labeled in centimeters. The scale of the slider is scaled 1:1 to the values it depicts. Participants have to use this element to report the height of the bridge via visually matching the slider to the bridge height. The tilting slant pad can be pivoted on one axis perpendicular to the station, forward or backward. The pad’s general design is inspired by physical devices described in the literature on geographical slant perception [21, 22]. Refer to Figure 4.13 for a comparison of slant measure capture apparatus used in related literature and our design. The top surface of the slant pad contains a high-contrast Figure 4.14: Reporting Station after crossing each bridge - Phase 1: Containing height reporting slider for scale visual input and slant pad for slant reporting. Left: Front view; Right: off-angle view with showing slant pad form. 41 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation striped texture to facilitate easier judgement of the tilt angle. Figure 4.14 showcases the elements of phase 1 of the reporting station. • Reporting station - Phase 2 : After completing Phase 1, the reporting station automatically switches to Phase 2. Phase 2 contains three vertical sliders for capturing subjective user sentiments. The sliders were set up akin to a 7-point Likert-like scale, with markings ranging from negative sentiments on the bottom, through neutral sentiments in the middle, to positive sentiments on the top. Input for each scale is encoded as a float value from +3 (for the most positive answer) to -3 (for the most negative answer). Figure 4.15 shows phase 2 of the reporting station. The height report arm and phase 1 of the reporting station feature a drawing to remind users about the data they have to provide. For further details on what tasks the participants were asked to perform and what input they were asked to provide, refer to Section 5.1). Figure 4.15: Reporting Station after crossing each bridge - Phase 2: Containing sliders for reporting subjective experience related to the previous bridge crossed. 4.2.5 Logging To facilitate the evaluation of the user study, a large amount of data is logged about the user’s behaviour in the virtual environment. This data includes: • Test conditions and study session information. 42 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.2. Software • The position and orientation of the users head (HMD) and body (tailbone tracker). • The height and pitch rotation manipulation values during the time spent on a bridge. • The time stamp when the user steps onto/off from a bridge and when redirection starts/finishes. From these, the duration of the time spent on the bridge and with completing the redirection is derived and stored as well. All logging scripts can be set to log data at every frame or at preset intervals. Different log files can be created from the same data in both human- and machine-readable format. The specifics about the encoding and recording of user input, refer to Section 5.3. 4.2.6 SteamVR Adaptation Our project uses the SteamVR Unity Plugin for tracking and stereoscopic rendering. However, during the implementations, we found that SteamVR exhibits problematic behaviour when the user repeatedly ascends/descends, i.e. steps onto the bridge and gets off again, and especially when going up/down the curved prop. These motions introduce inaccuracies into the tracking of the HMD, specifically into it’s vertical position. With repetition, this resulted in considerable drift, breaking bridge registration and causing false height perception. We found that this is caused by SteamVR trying to keep the registered floor height constant. We assume that it was not anticipated, that the HMD would regularly get to an elevated position while walking around. It is our assessment that SteamVR internally adjusts floor registration. It interprets the increases in height as tracking errors while walking on a flat floor, rather than actual uneven surfaces. Figure 4.16: Tripod with Vive tracker used to fix floor height in SteamVR. We countered this behaviour with the use of a second Vive tracker. We affixed the tracker to a tripod set to a measured height. Figure 4.16 shows our setup. At regular time intervals, we retrieve the raw tracking data for this tracker, and use the knowledge 43 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4. Implementation about its actual physical height to construct a SteamVR configuration object. This configuration object is then sent to the SteamVR plugin to effectively redo the floor registration portion of the room setup. Doing this registration every second allows us to keep the process imperceivable to the user, as the drift is still small enough and the correction does not cause a visible jump. Through these measures, we are able to ensures a fixed floor height. 4.2.7 Technical Setup Users are equipped with the HTC Vive Pro head-mounted display in a wireless configu- ration to allow the largest possible freedom of movement. The HTC wireless adapter battery is placed in a belt bag. Body location is tracked via an HTC Vive tracker in the tailbone position, worn on an elastic belt around the waist. Additionally, users are provided with two Vive Pro Controllers to allow task performance, support a more confident interaction with the physical bridge and facilitate the sense of embodiment. We use four 2nd generation HTC Lighthouse tracking stations to create a workspace of 6 x 10 meters. During preliminary testing, we derived the optimal Lighthouse placement to allow stable tracking during both the balloon popping game and the bridge crossing: Two Lighthouses are affixed to two opposite corners of the lab, while two additional Lighthouses are placed on tripods on each side of the bridge to mitigate occlusion of the tailbone tracker by the railings. The VE is rendered using the Unity 3D game engine on a desktop PC with an Intel Core i9-9900K CPU and an NVidia RTX 2080Ti graphics card. The audio transducer affixed to the bottom center of the physical bridges is a 400-1500w Buttkicker LFE. It is controlled from a notebook, connected to the transducer’s amplifier. 44 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . CHAPTER 5 Procedure Description To evaluate our hypotheses, we conducted two user studies. We outlined the study procedure carefully to ensure the following aspects: • Safety: At all steps of the procedure, the safety of study participants was of the highest priority. We set up the workspace, the physical bridge props, the supple- mentary material and the the verbal introduction to the ensure that participants understand the risks involved, and are aware of the ways these risks can be mitigated. At the time of conception and realization of the user studies (June-July 2019), the TU Wien did not have an active ethics board in place to review the planned procedure. Therefore, we followed recommendations for standards and practices from literature published on the subject to guarantee the health, safety and general ethical treatment of participants. This includes the protection of privacy and personal data collected during experimentation. Furthermore, we consulted the ethics advisor of the TU Wien on our procedure plans. Additionally, we conducted a pilot test with expert users to empirically evaluate procedure safety. • Invariance: We devised the study procedure to ensure that all participants receive the same clear and concise information to prevent misunderstanding, resulting in unreliable data capture. To this end, we conducted iterative preliminary testing to determine hard to understand or non-intuitive steps. The final procedure incorporated these findings to maximize participants’ understanding of the task they have to perform, and of the means to perform them. • Efficiency: From preliminary testing, we understood the demand the study placed upon participants and experimenters. We set up study procedure to be as effective as possible, with the goal of minimizing the required time and effort for users and experimenters alike. 45 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 5. Procedure Description This chapter contains details about how these main guidelines were implemented. Since we conducted two user studies with the two different physical bridge props, there are slight differences in study procedure. First, we provide a general description about circumstances common to both studies in Section 5.1. We describe the task participants were asked to perform in Section 5.2 and the measures we used in Section 5.3. 5.1 General Study Procedure For the successful creation of an illusion, the participants were not allowed see the workspace before the experiment. Participants were greeted outside the laboratory and briefed about health and safety, privacy and data protection details. They read and signed the informed consent form, confirming that they are physically fit to participate, and that they understood the risks and general recommendations for the VR exposure. Next, participants filled out Kennedy’s simulator sickness pre-test and the pre-test information questionnaire. After the pre-test questionnaire was completed, participants received a detailed briefing about the experimental procedure. The briefing included explanations of the following topics: • The virtual environment: We set the scene for the participants to allow them to get into an excited disposition about the experience. We described the ambiance of the mountain lake scene, and assigned the goal of proceeding from island to island until the last island is reached. • The task to be performed: Participants were asked to complete this goal by crossing the virtual bridges one by one and reporting their experience. They received a step-by-step explanation about the crossing cycle. They received instructions on how to use the UI and were also asked to verbally comment on their experience or ask questions while they are in VR. • Health and safety: Participants were asked to move carefully, using special caution when stepping onto / off from a bridge. Participants were briefed that they are allowed to take a break between tasks or discontinue the experiment at any moment if they wish. • The equipment: Participants were given an explanation about the purpose of each piece of equipment they will be using. They received instructions on when and how to set the interpupillary distance (IPD) on the HMD. They were shown where the buttons of the controller are located and how they can be used during the experiment. After the last step, the participants were fitted with the equipment and led to their starting position in the lab workspace. During this time, participants’ vision was occluded by the deactivated HMD to keep them naive to the lab arrangement. 46 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 5.2. Task Before performing the actual task, participant were instructed to set the IPD according to their preferences and to get acquainted with the VR controls. To allow participants to get used to the UI, we have created a dummy reporting station on the starting island. Using this dummy station as a guide, we repeated the instructions about the function of each UI element and let the participants try out each one. We commenced with the actual tasks only once participants reported being comfortable with the control scheme and the function of the UI elements. Once participants reached the last island, the task performance phase concluded. At this point we deactivated the HMD and led the participants out of the lab workspace. During this time, their vision was still occluded by the deactivated HMD. Outside lab the equipment was removed. Participants were asked to fill out the Kennedy SSQ post-test and the post-test questionnaire. Finally, participants were invited to inspect the lab workspace and share their impressions orally. 5.2 Task Our hypotheses require us to evaluate a considerable number of variable combinations. Therefore, a large number of bridges have to be crossed by each participant. As a consequence of this, the task that participants have to perform contains repetition, which requires them to spend a considerable time in VR. This introduces monotony, which in turn has the potential of lowering user motivation and attention. To alleviate monotony, we designed the repeated bridge crossing cycle as a game loop. In this game loop, the actual task, which is providing the data for our research, is interlaced with a gamified task that provides distraction for the user. Consequently, the user tasks can be broken up into three categories: • Perceptual component: Observing and crossing the bridge. • Data acquisition: Providing feedback input though the UI. • Distraction: A game of popping balloons that serves as a distraction from the monotony of the the main tasks, as well as a distraction for the redirection to take place. Our virtual environment consist of a series of islands connected by virtual bridges. Refer to section 4.2.1 for further details. The goal assigned to participants is to reach the last island by crossing each bridge. This goal can be achieved by repeatedly performing the following crossing cycle: 1. Observing the bridge from the island. All islands are visible from the beginning, but virtual bridges are spawned in one-by-one, as to not cause any distraction or confusion about which bridge is currently under evaluation. 47 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 5. Procedure Description 2. Carefully locating the bridge by touching the step with the tip of a shoe and the railings with the controllers. This is done to minimize the risk of injury. 3. Stepping onto the bridge and crossing it halfway. Participants are allowed to grab the railings if needed, but are asked to not slide the controllers along them. 4. Estimating the height of the bridge in centimeters. Participants have to input their estimation on the Height Reporting Arm and confirm it. To aid the estimation, participants can walk back and forth on the first half of the bridge if they want, but are not allowed to leave the bridge once they stepped on it. The Height Reporting Arm blocks the way forward until input is entered and confirmed. 5. Finishing crossing the bridge and approaching to the reporting station. 6. In Stage 1 of the reporting station, participants are asked to provide height and slant estimations for the bridge they just crossed. At this point, participants have to provide their measures visually (Refer to section 4.2.4 for details about the reporting station and section 5.3 for a summary of the measures we have inquired about). Here, we rely on the user experience as a whole, as an integrated sensory input from all sources. Therefore, at his stage, we disallow the user to inspect the bridge visually again. When the user leaves the bridge, the virtual bridge is destroyed, leaving only small parts from each end as a reference of its location. 7. In Stage 2 of the reporting station, participants are asked to report their subjective experience. Participants are asked about subjective their preference related to the bridge they just crossed. Refer to section 5.3 for further details about these measures. As soon as participants confirm their input in Phase 2 of the reporting station, the next virtual bridge appears, leading to the next island. However, at this point, the bridge is blocked by a wall. 8. Playing the balloon popping minigame. Participants are told they get to play the game as a reward, to make the whole experience more fun and interesting. During this time, the redirection takes place, as described in section 4.2.3. Participants have to continue looking for balloons until they hear a chime sound. At this time, the walls blocking the next bridge are removed. (The redirection is finished.) This crossing cycle continues until the participant reaches the last island, at which point the task performance phase concludes. 5.3 Measures To provide data for our analysis, we asked the participants to provide feedback via three different means: • Through input via the UI interfaces inside the VE. 48 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 5.3. Measures • Through pre- and post-test paper-based written questionnaire. • Through oral feedback any time during their participation. In the following we explain the characteristics of the measures we sought to acquire, the instructions participants received in order to correctly provide answers and the processing of the input they provided. The means by which participants provide measures in the VE is structure by the steps of the crossing cycle, as described above in Section 5.2. For a detailed technical description of the UI elements, refer to Section 4.2.4. In the VE, participants were asked to estimate the height and the slant (i.e. steepness) of each virtual bridge, and to share their subjective preference. On the bridge, we asked them to estimate the height in centimeters using a simple horizontal slider. We provided a range of possible input values that contained all simulated bridge heights. We encoded and logged the input as integer values in centimeters. On the island we used the Reporting Station. In Stage 1, participants were asked to estimate the height of the experienced bridge using a vertical slider. A corresponding scale contained markings only at zero and at maximum scale values in cm, and its height matched the real-world scale 1:1. Participants had to provide their estimation by grabbing the slider and positioning it at a height that visually corresponded to the maximum height of the bridge they just crossed. We encoded and logged the height as float values in centimeters. Then, we asked participants to tilt the slant measuring pad from a neutral horizontal position to an angle visually corresponding to the maximum slant they experienced on the bridge. The tilt angle input was encoded and logged as float values in degrees. Refer to Figure 5.1 for an illustration of the UI elements. After confirming their estimates, participants answered the following questions about their subjective experience of the recent bridge: • "How do you feel after this bridge?", relating to their general state. • "How did you like this bridge?", relating to personal preference concerning the bridge. • "How natural was the movement on the bridge?", relating to their perception about the naturalness of the experience. All answers to these questions were given via 7-point float Likert-like scaled sliders. The main points were annotated from very positive, through neutral, to very negative. We encoded the inputs as float values from +3 (for the most positive answer) to -3 (for the most negative answer). Participants filled out written questionnaires before and after the their VR experience. Before the test, participants filled out Kennedy’s simulator sickness pre-test [75] and a 49 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 5. Procedure Description Figure 5.1: Experience reporting interfaces inside the VE. Note the illustration reminding participants of the measure they are supposed to provide in the upper right corner of each element. Left: Reporting interface on the bridge as used in study 1 [-70cm...0cm...70cm]; Center: A reporting station on an island as used in study 1 [-70cm...0cm...70cm], close-up view of the height slider (left) and the slant estimation pad (right). The slant pad is already tilted from a neutral horizontal position upwards at 20 degrees; Right: Reporting interface after the bridges, full view as used in Study 2 [0cm..140cm]. general information questionnaire, screening for age, gender, conditions affecting vision (acuity, 3D vision capability, colorblindness), as well as gaming, HMD use and VR experience. After the experiment was completed, participants filled out the SSQ post-test and answered a the following additional questions: 1. "How did you estimate the height of the bridge?", relating to the mental process of arriving at their estimated value. We encouraged participants to provide details about what qualities and circumstance influenced their estimation. 2. "How many virtual bridges of different heights did you notice?" We encouraged participants to take time and recount the experience before providing a number. 3. "How many real bridges were used?" We inquired about the participants percep- tion about how many real bridge props of different heights were used during the experiment. 4. "Was there anything that contributed or took away from the experience?" We asked the participants to name any specific circumstance experienced during their time in the VE that they found to be notably pleasant or notably unpleasant / disturbing. Finally, participants could also leave any further comments they considered important. 50 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 5.3. Measures Additionally, participants were encouraged to provide oral feedback at any point during their participation about any details of the experiment they deemed important. We compiled written logs from the important remarks. Furthermore, consenting participants were recorded on video / audio during task performance. 51 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . CHAPTER 6 Evaluation This chapter contains details specific to user study 1 (flat physical prop) 6.1 and user study 2 (curved physical prop) 6.2, including diverging study design, population data, results analysis and discussion. 6.1 User Study 1 This study followed the general design and measures described above. Only one physical bridge was used to simulate different virtual bridges. 6.1.1 Study Setup For Study 1 we used the flat physical bridge prop (for construction details see Section 4.1). The level walkway surface of the bridge is equally elevated from the floor by 12 cm to ensure stability and accommodate structural elements (audio transducer and vibration conducting metal). Our virtual bridge model included this peculiarity so that participants saw that a step up should be taken for every virtual bridge. For providing measure input during task performance, we asked the participants to regard the step on the bridge (12 cm from the floor level) as a starting (zero) point of height and slant estimations. An illustration of how to do the estimations was used during the instruction process and placed on all the reporting interfaces in the VE. See Figure 6.1 for a description, and Figures 4.12 and 4.14 for an illustration of the placement un UI elements in the VE. We set up a VE with 17 virtual bridges to allow each combination of conditions to occur. Five levels of added visual heights have been selected in preliminary testing, with an interval of 20 cm for height manipulations. This formed a set of two hanging (concave) bridges with -40 cm and -20 cm added visual height respectively, and two upwards curved 53 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6. Evaluation Figure 6.1: The illustration placed on every UI element on the VE to remind the study participants about what measures they have to input. We marked the step on both side and labelled the height we asked participants to estimate. We also marked and labelled the slant angle for which participants had to provide an estimation. (convex) bridges with 20 cm and 40 cm maximum deviation from the flat surface of the real bridge. Finally, there was a flat bridge identical in shape to the physical bridge prop. This bridge acts as the ground truth in this setup. Refer to Figure 6.2 for a partial view of the VE and to Figure 6.3 for a showcase of the visual bridge heights used in the study. The sequences of bridges were generated using a balanced Latin square. Additionally, each bridge sequence was counter-balanced to compensate for the potential learning effect. Figure 6.2: Bird’s-eye partial view of the VE used in Study 1 with a flat physical bridge. The whole scene contains 17 virtual bridges that differ in visual heights and in the multi- sensory manipulations applied. Note, that during task performance the participants only saw one bridge at a time. 54 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6.1. User Study 1 Figure 6.3: Showcase of the different visual heights of the virtual bridges used in study 1. From left to right: -40 cm, -20 cm, 0 cm (ground truth, identical to physical bridge prop), 20 cm, 40cm. In every sequence, the flat virtual bridge was placed as the last one to cross. We chose this configuration because the flat bridge was too revealing for the setup and would have potentially biased any other virtual bridge coming after it. Moreover, the results of this bridge were problematic for the statistical analysis, since neither pitch manipulations nor height manipulations could be applied. The total time of the experiment was approximately 1 hour, including briefing, instructions, pre- and post-test questionnaires and task performance. 6.1.2 Population 22 persons took part in study 1: 10 female and 12 male, aged between 21-64 years with a mean of 38.5 years (standard deviation SD = 11.68). 12 participants (54.4%) indicated that they had no prior experience with VR. The others declared to have some experience, and only one participant indicated being a VR expert. Participants were recruited on a volunteer basis, via Facebook and a mailing list for VR research recruitment. They were required to be over 18 years old, not suffering from severe motion sickness, epilepsy or any other critical health condition, as well as from any contact-transmitted diseases. All of the participants had normal or corrected to normal vision. One person reported not being able to see 3D. 6.1.3 Results To analyze the data we mainly used a 3-way Factorial ANOVA. For the cases when the assumption of sphericity was violated, we report the F-values according to the 55 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6. Evaluation conservative Greenhouse-Geisser correction (ǫ). Contrasts and T-tests are done with the Bonferroni adjustment for multiple comparisons, where it is appropriate. For the quantitative measure of the strength of a phenomenon, we rely on the measure independent Pearson’s correlation coefficient (r) for effect size estimation and on Cohen’s benchmark for interpretation (r ≈ 0.1 - small, r ≈ 0.3 - medium, and r ≈ 0.5 - large effect). Height Perception We compared the height estimations made on and after the bridge for all tested cases. We found significant correlations (Rcorrelation > 0.5, p < 0.01) between the estimations made for all virtual bridges, except for the case with height of -20 cm and vibrotactile stimulation (Rcorrelation = 0.331, p = 0.132). However, the paired samples T-test was significant (significance threshold p < 0.025) only for the case with visual height of -40 cm and added camera pitch (t(21) = −2.733, p = 0.012, r = 0.5, Rcorrelation = 0.936). For the rest of the cases the differences were not significant (p > 0.1). Therefore, we relied on the data collected at the reporting stations for the analysis, as we deemed this input to be representative of the complete experience. A boxplot with the height estimation results clustered according to the multi-sensory simulations used is shown in 6.4. The sphericity test determined the assumption was violated for the height estimations (χ2(5) = 80.092, p < 0.001, ǫ = 0.371). There was a significant main effect of added visual height (virtual bridge model shape and added virtual camera height coupled) on height judgements made (F (1.112) = 34.373, p < 0.001). Other independent parameters did not produce any trends or statistically significant effects. Contrasts showed that the height estimations differed significantly between different added visual heights (all p < 0.001, Md stands for mean difference): • -40 and -20 cm (F (1) = 28.908, r = 0.76, Md = 10.44 cm), • -20 and +20 cm (F (1) = 30.142, r = 0.77, Md = 17.02 cm), • +20 and +40 cm (F (1) = 25.681, r = 0.74, Md = 11.66 cm). Slant estimations A boxplot with the slant estimation results clustered according to the multi-sensory simulations used is shown in Figure 6.4. The Maulchly’s test showed that the sphericity assumption was violated for the visual height variable (χ2(5) = 78.988, p < 0.001, ǫ = 0.381) as well as for the interactions of visual height and vibrotactile stimulation (χ2(5) = 12.643, p = 0.027, ǫ = 0.72) and visual height, vibrotactile, and camera pitch manipulation (χ2(5) = 13.462, p = 0.02, ǫ = 0.788). There was a significant main effect of visual height on slant estimations (F (1.144) = 35.763, p < 0.001). 56 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6.1. User Study 1 Figure 6.4: Study 1: Clustered boxplots for the height estimations (left) and slant estimations (right) for the flat physical bridge. Added visual height denotes the maximum additional height added to the virtual bridge model that replicated the real bridge. The estimations are done relative to the start of the bridge (12 cm off the ground). The slant graph includes the indication of the maximum slant of virtual bridges in accordance with the value of added visual height. 57 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6. Evaluation Contrasts showed that the slant estimations differed significantly between different visual heights in favor of the higher ones (unless stated otherwise p < 0.001): • -40 and -20 (F (1) = 17.762, p = 0.002, r = 0.68, Md = 6.3°), • -20 and +20 (F (1) = 30.821, r = 0.77, Md = 12.77°), • +20 and +40 (F (1) = 35.879, r = 0.79, Md = 6.5°). Subjective Experience Bridge Post-exposure Physical State Self-reports. Camera pitch manipulation had a main effect on the participants’ state self-report values with a large effect size (F (1) = 8.787, p = 0.008, r = 0.55). That suggests that pitch manipulation might slightly contribute to the cybersickness. After the bridges without pitch manipulation participants’ self-reports were slightly more positive than with it (Md = 0.26 of a point) on the 7-point float Likert-like scaled sliders ranging from +3 (for the most positive answer) to -3 (for the most negative answer). Bridge Preferences. Camera pitch manipulation also lowered the scores for bridge preference by 0.22 of a point on average, resulting in a trend for significance with a medium-sized effect (F (1) = 4.349, p = 0.05, r = 0.42). Bridge Naturalness. The visual height had a statistically significant main effect on this measure (F (3) = 13.766, p < 0.001). Contrasts revealed the significant differences in answers for the conditions with added visual heights. The bridges with lower amounts of added visual height (±20cm) were perceived as "somewhat natural", while conditions with ±40cm of added visual height were rated as "somewhat unnatural" if there was pitch or vibrotactile stimulation. Our baseline condition was rated the highest (MD = 2.29, SD = 1.05) which is equivalent to answer "natural" in comparison to the rest of conditions that had the median of ratings slightly below 1 in case of ± 20 cm and around or below 0 for ± 40 cm of added visual height. Refer to Figure 6.5 for a clustered boxplot visualization of the analysis results. Cybersickness As the numbers and strength of the reported symptoms were rather low, we computed the sum of the SSQ symptoms for each participant for pre - and post-exposure. Due to very warm weather, we had to exclude the symptom of sweating to avoid biasing the results, since a considerable portion of participants reported stronger sweating in the pre-test SSQ than in the post-test. The paired samples T-test showed that on average participants experienced a slight increase in total cybersickness score (Msumdiff = 1.0, SD = 2.07) which was statistically significant (t(21) = 2.266, p = 0.034, r = 0.44). That means that the SSQ post-exposure scores typically increased by 1 point for only one symptom. 58 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6.1. User Study 1 Figure 6.5: Study 1: Boxplots for the subjective experience questions for the flat physical bridge. 59 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6. Evaluation Number of Different Virtual and Real Bridges According to participants’ reports, they noticed a mean number of M = 7.18, SD = 2.79 virtual bridges that had different heights, even though there were only 5 distinct visual heights in the VE. For the real bridges, the mean detected number was M = 2.86, SD = 3.32. Paired samples T-test also showed that the difference in numbers of virtual and real bridges is statistically significant with a large effect size (t(21) = 8.116, p < 0.001, r = 0.87). 6.1.4 Discussion The use of the flat surface for uneven terrain simulation has its benefits and drawbacks. The positive factors are its practicality and lack of limitations for the type of simulated terrain (elevation or indentation). However, our results show that both simulated heights and slants were strongly underestimated in all conditions, suggesting that the persuasiveness of the simulations was not very high. The height estimations for the virtual bridges were equal to a half of the simulated visual height if the virtual bridge was convex, and even less for the concave bridges. The severe underestimations cannot be explained by the assumption that participants had misunderstood the task, i.e. using the ground as baseline instead of the first step on the bridge. If this would be the case, it should reflect in all estimations for the virtual bridges, including the ground truth flat bridge, and shift them up uniformly by at least 10 cm. Consequently, an expected height estimation from the ground for the flat bridge would be 12 cm, but it follows the task instructions and is estimated to be 0 cm. Furthermore, this would increase the estimations for the upward curving bridges, which is also not the case. Therefore, the fact that neither of these happened suggests that the task was understood correctly. The conditions with vibrotactile stimuli showed a tendency to bring the height estimations slightly closer to the desired effect for the large added visual heights (i.e. ± 40 cm), but not for the smaller values (refer to Figure 6.4, left boxplot). That might be linked to a potential of vibrations to introduce noise to the proprioceptive, tactile and vestibular systems, causing a re-weighting of sensory cues and facilitating visual-like capture when the sensory conflict is resolved. Further research is necessary to draw a conclusion. Similarly, the slant estimations were lower than the expected values derived from the virtual bridge visuals. The estimations were closer to the expected values for the upward curving convex bridges and the additional stimuli seem to contribute to an increased slant estimation. In the case of the hanging bridges, the underestimation is noticeably stronger and correlates with the height estimations. One possible explanation for this can be hypothesized to be rooted in the difference between trying to step up onto an elevation and then "falling through" to the flat surface, versus trying to step down into an indentation but being blocked by the flat surface. In the latter case, we observed dismissive user reaction more often, where participants would complain about something "not being right", "being unexpected", etc. We assume that the exact distance the 60 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6.2. User Study 2 foot travels upwards, then downwards, is harder to judge precisely than the immediate resistance of the flat physical surface against trying to step downwards into a virtual indentation. Another explanation is that the upward curving bridge is more plausible in a lab workspace than a hanging configuration. One of the participants reported in the post-test conversation that he knew the hanging bridges were not possible, since "we haven’t climbed up onto anything when we entered the lab". Unlike the research conducted on real world height estimation [29], in our setup the height estimations did not depend on whether the observation was made looking down from the top of the bridge or standing on the ground. Furthermore, participants were able to distinguish the original bridge from the visually similar bridge that had the lower amount of added visual height applied (20cm). It was rated as the most natural and both estimations for it had the smallest variation. Moreover, almost half of the participants at the end of the experiment knew that there was only a single physical bridge height. We would expect an even higher detection ratio if no prop is used for the simulation. In summary, our results suggest that simulations of smooth uneven surfaces might benefit from the use of non-flat physical surfaces. If a flat physical surface is used for the uneven terrain simulation, the minimum amplitude of the visual height variation should exceed 30cm and possibly employ vibro-tactile stimulation. 6.2 User Study 2 In our hypothesis H1, we presume that the haptic and proprioceptive sensations associated with actual vertical traversal will allow for a larger manipulation. Driven by this assumption, we built a curved physical bridge prop in the hopes of achieving better simulation results than in study 1. The curved bridge prop has the same parameters as the bridge in study 1 (with a metal construction underneath), but the walkway is arched upwards over the ground. In study 2, only the curved prop is used. 6.2.1 Study Setup Unlike the flat prop used in study 1, the curved physical bridge allowed pitch manipulation for the ground truth. Furthermore, it inherently required physical vertical traversal. Thus, it also did not expose the manipulations and could be included anywhere in the experimental sequence. For the maximum height of the prop, we tried 20 and 30 cm. As the former did not result in a well perceivable slant, we decided to use 30 cm height. For safety reasons, the shape of the bridge was made a bit more flattened, and not as curved as the virtual bridges in study 1. Therefore, the bridge had a slightly lower step on each end as well. We adjusted the 3D model’s parameters to match the physical bridge prop. This resulted in a slightly different but still comparable real and virtual slants than in study 1. 61 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6. Evaluation In study 2, we maintained the 20 cm intervals for the added visual bridge height. However, this time we excluded the hanging bridges due to the risk of tripping and falling, and the intense conflict between the visual and the haptic feedback. Still, we kept one negative height adjustment value, which in the case of the curved prop resulted in a virtual bridge that was still upward curving, but had a lower height than the physical prop. These considerations lead to the following set of added visual heights: -20 cm, 0 cm (physical bridge height unmodified), +20 cm, and +40 cm. Considering the aforementioned 30cm maximum height difference in the walkway of the physical bridge prop, this resulted in virtual bridges with the following heights: 10 cm, 30 cm (ground truth, identical to physical bridge prop), 50 cm, 70 cm. The binary variables (vibration and pitch) were introduced in the same way as in study 1. This configuration resulted in 16 different virtual bridges. The counter-balanced sequences were generated via a balanced Latin square. Measuring methods, technical setup and procedure were identical to study 1. A partial view of the VE is shown in 6.6. Refer to Figure 6.7 for a showcase of the visual bridge heights used in study 2. Just as in study 1, the experiment required approximated 1 hour for each participant. 6.2.2 Population Participants were recruited in the same manner and under the same conditions as for study 1. 21 persons participated in study 2: 11 females and 10 males, aged between 23-63 years with a mean of 34.43 (SD = 12.24). Nobody among them participated in study 1, as we explicitly recruited new participants to avoid prior knowledge of the experimental setup. 11 participants (52.4%) indicated that they had no prior experience with VR. The Figure 6.6: Bird’s-eye partial view of the VE used in Study 2 with one curved physical bridge. The whole scene contains 16 virtual bridges, with visual heights of 10/30/50/70 cm. Note, that during the study the participants saw only one bridge at a time and there were no visible bridge during the task performance. 62 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6.2. User Study 2 Figure 6.7: Showcase of the different visual heights of the virtual bridges used in study 2. From left to right: 10 cm (with -20cm height adjustment), 30 cm (ground truth, identical to physical bridge prop, without height adjustment), 50 cm (with +20 cm height adjustment), 70 cm (with +40cm height adjustment). rest declared to have some experience. Once again, only one participant reported being a VR expert. 6.2.3 Results Results are reported using the same methods and data encoding as in study 1. Height Perception A boxplot with the height estimation results clustered according to the multi-sensory stimulation component(s) used is shown in Figure 6.8 on the left. The sphericity test determined that the assumption was violated for the height estimations (χ2(5) = 31.083, p < 0.001, ǫ = 0.5). There was a significant main effect of added visual height on height judgment made after experiencing the bridge (F (1.5) = 24.693, p < 0.001). Contrasts showed that the height estimations differed significantly between different added visual heights (p < 0.008): • -20 and 0cm (F (1) = 12.94, r = 0.62, Md = 5.99cm), • 0 and 20cm (F (1) = 22.62, r = 0.73, Md = 9.18cm), • 20 and 40cm (F (1) = 8.8, r = 0.55, Md = 9.21cm). Slant Perception A boxplot with the slant estimation results clustered according to the multi-sensory stimulation component(s) used is shown in Figure 6.8. Maulchly’s test showed that the sphericity assumption for the slant estimations was violated for the visual height 63 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6. Evaluation Figure 6.8: Study 2: Clustered boxplots for the height estimations (left) and slant estimations (right). Height estimations have additional indication of visual heights of the bridges in the VE. Slant graph includes the indication of the maximum slants of the 3D models for each level of added visual height. 64 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6.2. User Study 2 variable (χ2(5) = 22.859, p < 0.001, ǫ = 0.605). There was a significant main effect of visual height on slant estimations (F (1.818) = 31.512, p < 0.001) and slant adjustment on slant estimations (F (1) = 7.056, p = 0.015). Contrasts showed that the slant estimations differed significantly between different visual heights in favor of the higher ones: • -20 and 0cm (F (1) = 11.317, p = 0.003, r = 0.6, Md = 2.62°), • 0 and 20cm (F (1) = 22.78, p < 0.001, r = 0.73, Md = 4.18°), • 20 and 40cm (F (1) = 11.478, p = 0.003, r = 0.6, Md = 4.99°). Subjective Experience Bridge Post-exposure Physical State Self-reports. There were the following effects on the participants’ state self-report values: statistically significant main effects of added visual height that violated the sphericity assumption (χ2(5) = 17.803, p = 0.003, ǫ = 0.665, F (1.995) = 13.668, p < 0.001), and interaction of added visual height and pitch manipulation (F (3) = 4.329, p = 0.008). Pitch manipulation alone resulted in a trend for significance (F (1) = 4.068, p = 0.058), although its presence decreased the mean rating by 0.2. Contrasts revealed the significant differences for the following levels of added visual height: -20 and 0 cm (F (1) = 6.99, p = 0.016, r = 0.51), as well as 20 and 40 cm (F (1) = 21.066, p < 0.001, r = 0.72). Bridge Preference. Unlike in study 1, we found the following statistically significant effects for the preferences for bridges: added visual height (F (3) = 21.461, p < 0.001, r = 0.73), camera pitch (F (1) = 8.741, p = 0.008, r = 0.56), and their interaction (F (3) = 4.585, p = 0.006, r = 0.44). In addition, there was a trend for significance for the interaction of added visual height and vibrotactile stimulation (F (3) = 2.741, p = 0.052, r = 0.35). Bridge Naturalness. For the naturalness, statistically significant main effects were found for the added visual height (F (3) = 30.659, p < 0.001, r = 0.79), the pitch manipulation (F (1) = 6.314, p = 0.021, r = 0.5), the interaction of added visual height and vibrotactile stimulation (F (3) = 3.946, p = 0.013, r = 0.41), and only a trend for the interaction of added height and pitch manipulation (F (3) = 2.575, p = 0.063, r = 0.34). The naturalness rating differed significantly for all the visual heights (p < 0.004). The bridges without height manipulation were rated as the most natural M0cm = 1.585, and the bridges with added visual height were rated as "somewhat unnatural". The introduction of vibrotactile stimulation made a difference for the added visual heights -20 and 0 cm (F (1) = 7.431, p = 0.013, r = 0.53), as well as for 0 and 20 cm (F (1) = 6.228, p = 0.022, r = 0.5), increasing the scores for the cases with height manipulation. Number of Different Virtual and Real Bridges According to participants’ reports, they noticed 5 (M = 5.33, SD = 2.37) different heights of virtual bridges, which is once again higher that the actual number (4). As for the real bridges, participants reported 4 (M = 3.9, SD = 4.31) real bridges on average. 65 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6. Evaluation Cybersickness As the cybersickness symptoms were barely present, we used the same approach as in study 1: We summed up all the symptoms, excluding sweating. The mean total change in state among the participants was only slightly higher than in study 1 (Msumdiff = 1.9, SD = 2.23), and was statistically significant (t(20) = 3.907, p = 0.034, r = 0.66). That means that the SSQ post-exposure scores typically increased by 1 point for two symptoms or by 2 points for one symptom. 6.2.4 Discussion We observed major height overestimation for the bridges without height manipulation and with negative added visual height. The underestimation was observed only for a large added visual height. However, it seems that the underestimation can be countered via a combination of pitch and vibrotactile manipulations, bringing the estimations in accordance with the visual appearance of the virtual bridges. Similarly to height, for the virtual bridges with added visual height of -20 cm and 0 cm, there was an overestimation for the slant as well. For the bridges higher than the physical prop, the slant was underestimated. Except for the case with negative added height, a combination of pitch and vibrotactile stimulation noticeably improved the simulation, bringing the estimations closer to the values suggested by the virtual bridge visuals. The state self-reports suggest that participants could feel if the added pitch did not correspond to the physical bridge, even though the added camera pitch rotation was scaled back to half the angle of the visual slant. At the same time, the vibrotactile stimulation had no such negative effect and somewhat improved the preference and naturalness scores for the virtual bridges that were higher than the physical one. However, this only happened when used without the pitch manipulation. For the virtual bridges without added height, conditions with vibrations were somewhat less appreciated and natural. Our results suggest that uneven real surfaces can be simulated using the slightly curved real props. However, the prop surface orientation should not conflict with the presented visuals. In other words, the visual height might be added only in the direction matching the bend of the real surface. Moreover, a combination of pitch rotation and vibrotactile stimulation can be used to create a more convincing illusion of uneven virtual surfaces that otherwise clearly exceed the height of the real prop. However, the post-exposure answers suggest that the amount of the added pitch to the virtual camera should be slightly lower than the 50% of the slant of the virtual bridge that we used. Further research is needed to check if the same would apply to the concave surfaces, and for determining the optimal scaling of the pitch rotation angle. 66 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . CHAPTER 7 General Discussion, Conclusion and Future Work In this final chapter, we provide a summary of the research we conducted in the course of this thesis projects. In Section 7.1, we summarize the discussion of the two user studies and elaborate the consequences of our findings for our research hypotheses. In Section 7.2 we outline our research topic from inception through realization to evaluation. In Section 7.3 we discuss additional ideas and new findings that could be expanded upon in future continuation of this work. 7.1 General Discussion Our findings suggest that simulation works best with a virtual surface that reflects the real surface or raises above it. Although the use of a flat real surface allows for simulating a concave surface as well, our observations suggest that the illusion of indentation is less convincing than that of a protrusion. We assume that this is because the inability of lowering one’s feet into the nonexistent indentation is easier to detect. In contrast, the the height of the "fall-though" when trying to step onto a nonexistent bump is more difficult to judge precisely. Furthermore, the flat surface eventually will be detected by most users due to discrepancy between the visuals and the proprioceptive cues. The complete lack of physical effort to raise the own bodyweight up on a surface proves to be a highly revealing factor. These factors limit the amount of visual manipulation that might be applied and still be safe for retaining the illusion. For the surfaces with the added visual height in the range ±20cm, the height manipulation alone is sufficient. For the stronger height manipulations, the use of vibration seems to contribute to the height perception. Regardless of stimuli, the simulated slant tends to be greatly underestimated. 67 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 7. General Discussion, Conclusion and Future Work When using a real uneven physical surface, simulating concave surfaces (hanging bridges) is not possible, due to the strong disorienting effect. The attempts to simulate height and slant lower than those of the actual height of the convex physical bridge are not very successful either. In these cases, the haptic and proprioceptive cues seem to overrule the visuals. Of the conflicting sensory cues, the visuals have a higher variance, and are interpreted as faulty information. Our results suggest that in such cases height and slant perception exhibits haptic / proprioceptice (i.e. non-visual) capture-like behavior. This observation is consistent with the prior research by Ernst et al., where haptic feedback was reinforcing the visual slant perception if they were consistent and overruled it if the discrepancies crossed a certain threshold [42]. As mentioned above, using a convex physical surface to simulate a concave one puts too much stress on the the user. We evaluated such cases in our preliminary testing by trying to walk over a virtual bridge that was hanging down well below the floor level while using the convex physical bridge. Although is was possible to complete, the crossing resulted in a state of strong disorientation, similar to the one experienced during a roller coaster ride. We find these observations interesting to note, as the roller coaster involves purposeful overstimulation of vestibular apparatus linked with correct visual feedback, while our attempts were deliberately set up for a sensory conflict between the vision and proprioceptive and haptic sensations. Yet, the resulting experiences are quite comparable. The usage of the curved (convex) surfaces might be indeed beneficial for the simulations of uneven surfaces such as hills or bridges. Our participants systematically overestimated the height of the virtual bridges without height manipulation and correctly estimated the height of the virtual bridge 20 cm (66%) higher than the physical prop with height adjustment alone. An additional 40 cm (133%) to the prop’s height was perceived for the case where all three types of manipulation were used. That was not the case in study 1, therefore confirming our hypothesis H1. We assume that the usage of concave physical and virtual surfaces for depth simulations will produce similar results. However, further research is needed to confirm that. We have only partial evidence regarding the effects of pitch rotation manipulation (H2) and vibrotactile stimulation (H3). Neither produced a main statistically significant effect on height or slant estimations, but influenced the cybersickness, preferences and naturalness scores. In both studies, the conditions with pitch manipulation had increased variance of the perceived height and slant in comparison to the conditions without it. This manipulation also had a consistently negative impact on participants’ state self-reports, preferences, and naturalness, even though we scaled it down to 50% of the calculated value. Thus, we could not fully confirm our hypothesis H2. We assume that in a real world scenario, the actual pitch rotation created by walking on a physical uneven surface gets compensated via a complex interaction of lower body muscles and joints, and by the integration on proprioceptive and visual cues. We hypothesise that a much more complex process of calculating and introducing pitch rotation manipulation could lead to better results for both the simulation thresholds and the subjective user preferences. However, further research is needed to verify this assumption. 68 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 7.2. Conclusion Vibrotactile stimulation improved the height and together with pitch manipulation also the slant estimations for conditions with large added height (+40cm). It also seemed to improve the naturalness and preference scores for the bridges with high amounts of added height in study 2. At the same time, it was not effective for small values of height manipulations (±20cm). Therefore, the hypothesis H3 is only partially confirmed. Ultimately, our results suggest that use of curved physical props in combination with redirection and multi-sensory stimulation is a viable approach towards allowing natural walking in non-flat VEs. 7.2 Conclusion Smooth transitions between different heights are an inherent part of natural locomotion in the real world, but are largely missing from VR experiences. In this thesis, we address this issue by proposing and evaluating a simulation of smooth uneven surfaces in VR. The simulation uses a multi-sensory stimulation approach. We combine visual stimulation via camera viewport manipulations, passive haptics via physical bridge props and stimulation of the vestibular and proprioceptive systems via active vibrotactile stimulation. We select and design our stimulation components based on different stimulation techniques described in related works. From a review of previous research about height and slant perception and sensory integration, the cognitive and sensory processes that are relevant for an uneven surface simulation are derived. Our approach is created via a combination of previously proposed manipulation and stimulation techniques. Previously suggested camera viewport height and pitch rotation manipulations are utilized to visually simulate protrusions or indentations. Our approach develops these techniques further to allow them to be used with both flat and curved real physical surfaces. We investigate techniques on passive haptics in the context of VR and especially natural vertical locomotion in VR. Based on these insights, physical props are designed and constructed to evaluate the effect of different physical surface characteristics on the believability of the simulation. We review previous research on sensory integration theory in the context of height and slant perception. As a result, we concentrate on proposed stimulation of the vestibular and proprioceptive systems aimed at eliciting visual sensory capture in ambiguous sensory situations. From these insights, a vibrotactile stimulation scheme is devised, which is transmitted through our bridge props to introduce whole-body vibrations. We review previous work on the sense of presence and set our goal to create a simulation which does not break it, and which provides a high degree of immersion in the VE. Our main hypothesis states that an actual physical uneven surface will allow for a higher manipulation threshold without breaking immersion (H1). Our second hypothesis assumes that combining simulated visual cues for both height and pitch rotation will lead to a stronger simulation effect (H2). Our last assumption is that introducing noise into the vestibular system via vibrotactile stimulation will allow for an even stronger simulation effect (H3). To test our hypotheses, a complex software application is developed that allow us to precisely control the manipulation parameters. The application facilitates the 69 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 7. General Discussion, Conclusion and Future Work creation of virtual bridges with parametrizable model visuals. We build VEs where a sequence of multiple virtual bridges can be evaluated without introducing bias or learning effects. With the help of our application, we conduct two user studies: one with a flat physical bridge and another with an upward curving (convex) bridge. To enable the studies, we create supplementary systems, such as UI interfaces for data capture and a highly effective redirection method allowing the use of a single physical bridge prop for each study. By analyzing and evaluating the data collected in the course of the two user studies, we are able to confirm our main hypothesis H1. H2 and H3 are only partially confirmed by our results. 7.3 Future Work In the course of our project, different ideas were developed that could not be realised and evaluated due to technical, financial and temporal limitations. Furthermore, the evaluation of the two user studies led to the formulation of additional research questions. This section contains an overview of these ideas and questions, paired with propositions for further exploration possibilities in future work. There related literature suggests that providing the VR user with a higher degree of body representation (i.e. self-avatar) can lead to a stronger sense of presence and a sense of embodiment. Specifically for our research context, Asjad et al. [70] showed that having a representation of the user’s feet in VR adds to the illusion of vertical locomotion. Lenggenhager et al. [40] showed that the location where the stimulus was observed can override the visual perspective of self-location, when it is reinforced with congruent multi-modal stimulation. Therefore, we assume that our simulation would have benefited from feet tracking and feet representation. This is further supported by the observation that our manipulations used to keep the controllers aligned with the virtual bridges’ railings have not been rejected by users. However, feet tracking has proven to be too unreliable for our purposes. This was due to the occlusions that the physical bridges’ structural elements introduced in the tracking. We assume that with the advancement in tracking technology, or with the construction of a more expensive, thus less constrained bridge prop, the introduction of this feature would be possible, and would merit further investigation. Our conclusions about the effects of vibrotactile stimulation might have been influenced by the point of introduction. We decided to introduce whole-body vibrations though the entire surface of the physical bridge props to facilitate a less intrusive, more inconspicuous user experience. Meanwhile, previous vestibular stimulation techniques mainly rely on bone-conducted vibrations, introduced directly into the skull via head-attached vibrators. The point of introduction further limited us in the frequency range we could use, as the frequencies suggested in the related work for maximum vestibular response (around 100 Hz) were too faint when conducted through our wooden prop surface. Future research could trade the setup we used now for a more intrusive head-mounted vibrator setup, as suggested e.g. by Peng et al. [56]. 70 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 7.3. Future Work Out physical bridge props were constructed while adhering to temporal and financial constraints. We had to work around the shape of our curved prop, which turned out less than ideal. As mentioned above, we had to omit feet tracking due to a high degree of occlusion by the bridge structures. This in turn could not be avoided, as we also had to provide enough support for health and safety reasons. With more cost-effective materials this meant stricter constraints in design. A revised, more expensive design could potentially be more fine-tuned, especially when incorporating the insights gained from our two user studies. Additionally, as mentioned in Section 4.1, we designed a physical bridge prop with variable height. Refer to Appendix A for a detailed description and plans. Constructing this prop would allow further investigation of the simulation thresholds and their relation to the shape of the actual physical surface. Simulating smooth uneven surfaces in virtual reality poses significant challenges. Safety concerns, cybersickness and the potential breaking of immersion make a realistic simula- tion difficult. However, our findings suggest that a multi-sensory approach can be used to create such a simulation under specific circumstances. Constructing appropriate physical devices and designing the associated VR interactions can mitigate the limitations of current VR setups, adding another dimension to the experience. 71 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . APPENDIX A Variable Height Physical Bridge Prop 73 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . Adjustable Height, Lightweight, Low Cost Bridge Concept Balint Kovacs (01227520) A. Variable Height Physical Bridge Prop 74 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . Bridge concept for an easy to build (almost identical to a simple flat bridge), low cost adjustable bridge. The main idea is that the walkway(1) is supported by heavy duty truck cargo fastening belts/straps(2). The planks of the walkway are stapled to the strap (akin to a rope ladder/hanging bridge), the supporting girder parts(3) can be interchanged to configure the bridge to different heights. The strap is fastened in place, with a proposed strap duty of 2000 kg easily supporting an adult person. Some bending/flexing is to be expected, but the straps used for securing heavy cargo are strong, durable and non-elastic, minimizing the risk. The handrails(4) consist of lighter duty cargo straps, and auto- matically follow the form of the walkway. The bridge is lightweight and easily transportable. The bridge concept is based around following requirements: 1. low cost (< EUR 100) 2. easy to build (< 1 day) 3. configurable height 4. transportability 5. effective vibration distribution 1. The bridge is constructed only from two types of widely available lumber stock. Both are easy to obtain, cheap and easy to work with. The base frame consists of 100x100mm wooden girder. The railing posts and the walkway consist of 50x30mm stock. 2. The build is manageable in less than 1 day (8-10 hours). Only simple straight cutting, drilling, (some minimal) sand- ing, screwing/bolting and pneumatic stapling are required. (KB has all tools necessary.) Every work step can be managed with transportable power tools. Apart from the cargo straps, the workload is expected to be the same as building a flat bridge. 3. The height and form of the bridge can be configured by interchanging the supporting girder parts and re-fastening the strap. To incorporate extreme cases (flat/very high bridge), the last few walkway planks can be made detachable, allowing the removal/addition of planks as needed. 4. The bridge mostly consist of softwood, making its weight manageable. The walkway can be rolled up for transport, the handrail posts removed and the handrail straps used to bundle the material. For additional portability, the main base girders can be cut in two and made quick attachable with bolts, bringing the maximum transport length down to 1.4 m. 5. The buttkicker mounted on the central connecting element should allow the vibrations to travel into the two main girders. From there the vibration should be passed along into the supporting girder parts and into the walkway. Vibra- tions on the walkway midway between two supporting girders are expected to be lower. (1) walkway (2)walkway straps (4)handrail straps (3)interchangeable supporting girder parts 75 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 50x30 mm wood stock 100x100 mm wood girder 30 mm transport strap (e.g. 4m, 800kg pulling strength) 40 mm transport strap (e.g. 4m, 2000kg pulling strength) steel fastening rings buttkicker Cost Calculation Consisting of hungarian prices converted to EUR. 100x100 girder amount dependent on number of planned configurations. Material Amount Unit Cost (EUR) 50x30mm wood stock 26 m 13 100x100mm wood girder 9.9* m 29 30mm transport strap (6m, 1000kg) 2 pcs 10 40mm transport strap (4m, 2000kg) 2 pcs 12 Steel fastening rings 8 pcs 4 Screws, nuts, bolts, etc. 15 Total Cost 83 A. Variable Height Physical Bridge Prop 76 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 77 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . List of Figures 2.1 Slant perception experimental views using real and virtual environments from Proffitt et al. [22] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 Participants’ view in the tool embodiment experiment from Alzayat et al. [38]. Note the lack of full-body representation, which is not a hard requirement for the sense of embodiment to form. . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Illustration of vestibular stimulator prototype from Peng et al. [56]. They report the best results with the depicted 2-sided vibrator placement setup. 13 2.4 Table summarizing the taxonomy of redirection techniques, including some examples from Suma et al. [63] . . . . . . . . . . . . . . . . . . . . . . . . 16 2.5 Illustration of the built-in capabilities of the Redirected Walking Toolkit for Unity from Azmandian et al. [64]: Views and visualizations (left), inspector configuration parameters for the redirection (right). Configurable parameters include upper and lower bounds for translation and rotation gains, radius for calculating curvature gain and custom head transform object reference. . 17 2.6 Overview of all the effects tested by Bruder et al. [65]. They propose different visual effects for creating a self-motion illusion: "(a) particles, (b) sinus gratings, (c) and textures fitted to the scene, respectively (d) contour filtering, (e) change blindness and (f) contrast inversion. Illusory motion stimuli are limited to peripheral regions." . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.7 Illustration of the camera manipulations applied by Marchal et al. to simulate walking on an uneven surface [13]. . . . . . . . . . . . . . . . . . . . . . . 19 2.8 Illustration and demonstration of infinite virtual stairs via passive haptics from Nagao et al. [14]. Angled bars are placed on the floor equally spaced. The haptic sensation of stepping on the bars combined with manipulating the height of the virtual camera creates an illusion of ascending/descending stairs. 21 2.9 Electro-mechanical solutions for uneven surface simulation in VR. Gait Master from Iwata et al. [17] (left) and 6 DOF parallel manipulators from Yoon and Ryu [18] (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.10 Virtual environment and vibrotactile elevator simulation device from Va- sylevska and Kaufmann [71]. . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.1 Illustration of the flat bridge prop. Top: CAD drawing of the finalized design, used to evaluate stability and cost efficiency; Bottom: Finished construction of the flat bridge prop. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 79 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.2 Left: Audio transducer mount on the flat bridge prop: Underside of the walkway with aluminum rail; Right: audio transducer mounted on metal arm. 27 4.3 Flat bridge disassembled into components for transport and storage. . . . 27 4.4 Illustration of the finished curved bridge prop. . . . . . . . . . . . . . . . 28 4.5 Supporting welded steel arch structure and audio transducer mount on the underside of the curved bridge prop. . . . . . . . . . . . . . . . . . . . . . 28 4.6 Birds-eye view of the VE we created for our user studies. The virtual bridges depicted are of the type we used for the flat physical prop in our user study 1. Top: Overview of the entire VE with the surrounding mountainside and lake; Bottom: Close-up partial view of the VE with islands connected via differently configured virtual bridges, arranged in a consecutive crossing pattern. . . 30 4.7 Construction of virtual curved bridges. From top to bottom: Real curved prop for reference. Observe the artifacts in the walkway and the railing profile created by our limited resources during construction; Sampled walkway profile stored as Unity animation curve; Sampled railing profile stored as repurposed Unity animation curve. Both curves are depicted not to scale, but stretched in height to illustrate that they are the result of sampling points on the physical bridge walkway and railing for their height.; Virtual bridge created by using the repurposed animation curves. We are sampling them at the location each element or the virtual bridge model to create proportionate visual representations in the VE. . . . . . . . . . . . . . . . . . . . . . . . . 31 4.8 Position of the tailbone tracker used to calculate location of the user in the VE. This location data is used to determine the user’s position on the bridge and calculate camera manipulations. . . . . . . . . . . . . . . . . . . . . . 33 4.9 Illustration of camera manipulations used. . . . . . . . . . . . . . . . . . . 34 4.10 Illustration of how balloon spawn points are determined by observing optimal redirection potential and physical / virtual spatial constraints. . . . . . . 37 4.11 Demonstration of redirection introduced during balloon pop. Left: Balloon intact; Center: balloon popped, confetti effect; Right: the two frames overlaid to visualize the rotation introduced at the moment when a user touches a balloon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.12 Reporting arm on the bridge, used to inquire a numeric height value. Blocks bridge until input is confirmed. Left: Depicted as used on the flat bridge, with zero value in center, -/+50 cm extreme values; Right: Depicted as used on the curved bridge with a scale between 0 and 100 cm, depicted while manipulated by user with laser. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 4.13 Comparison of experimental apparatus used to capture participant feedback about perceived slant by Proffitt et al. 1995 [21] (left), Proffitt et al. 2001 [22] (center) and the one we created for our user studies (right). . . . . . . 41 4.14 Reporting Station after crossing each bridge - Phase 1: Containing height reporting slider for scale visual input and slant pad for slant reporting. Left: Front view; Right: off-angle view with showing slant pad form. . . . . . . . 41 80 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 4.15 Reporting Station after crossing each bridge - Phase 2: Containing sliders for reporting subjective experience related to the previous bridge crossed. . . 42 4.16 Tripod with Vive tracker used to fix floor height in SteamVR. . . . . . . . 43 5.1 Experience reporting interfaces inside the VE. Note the illustration reminding participants of the measure they are supposed to provide in the upper right corner of each element. Left: Reporting interface on the bridge as used in study 1 [-70cm...0cm...70cm]; Center: A reporting station on an island as used in study 1 [-70cm...0cm...70cm], close-up view of the height slider (left) and the slant estimation pad (right). The slant pad is already tilted from a neutral horizontal position upwards at 20 degrees; Right: Reporting interface after the bridges, full view as used in Study 2 [0cm..140cm]. . . . . . . . 50 6.1 The illustration placed on every UI element on the VE to remind the study participants about what measures they have to input. We marked the step on both side and labelled the height we asked participants to estimate. We also marked and labelled the slant angle for which participants had to provide an estimation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 6.2 Bird’s-eye partial view of the VE used in Study 1 with a flat physical bridge. The whole scene contains 17 virtual bridges that differ in visual heights and in the multi-sensory manipulations applied. Note, that during task performance the participants only saw one bridge at a time. . . . . . . . . . . . . . . . 54 6.3 Showcase of the different visual heights of the virtual bridges used in study 1. From left to right: -40 cm, -20 cm, 0 cm (ground truth, identical to physical bridge prop), 20 cm, 40cm. . . . . . . . . . . . . . . . . . . . . . . . . . . 55 6.4 Study 1: Clustered boxplots for the height estimations (left) and slant esti- mations (right) for the flat physical bridge. Added visual height denotes the maximum additional height added to the virtual bridge model that replicated the real bridge. The estimations are done relative to the start of the bridge (12 cm off the ground). The slant graph includes the indication of the maximum slant of virtual bridges in accordance with the value of added visual height. 57 6.5 Study 1: Boxplots for the subjective experience questions for the flat physical bridge. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 6.6 Bird’s-eye partial view of the VE used in Study 2 with one curved physical bridge. The whole scene contains 16 virtual bridges, with visual heights of 10/30/50/70 cm. Note, that during the study the participants saw only one bridge at a time and there were no visible bridge during the task performance. 62 6.7 Showcase of the different visual heights of the virtual bridges used in study 2. From left to right: 10 cm (with -20cm height adjustment), 30 cm (ground truth, identical to physical bridge prop, without height adjustment), 50 cm (with +20 cm height adjustment), 70 cm (with +40cm height adjustment). 63 81 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . 6.8 Study 2: Clustered boxplots for the height estimations (left) and slant estima- tions (right). Height estimations have additional indication of visual heights of the bridges in the VE. Slant graph includes the indication of the maximum slants of the 3D models for each level of added visual height. . . . . . . . 64 82 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . Bibliography [1] I. E. Sutherland, “A head-mounted three dimensional display,” in Proceedings of the December 9-11, 1968, fall joint computer conference, part I, pp. 757–764, 1968. [2] F. Technologies, “Oculus homepage.” https://www.oculus.com/?locale=en_ US. [Online; accessed: 25.04.2020]. [3] HTC, “Htc vive homepage.” https://www.vive.com/eu/. [Online; accessed: 25.04.2020]. [4] M. Usoh, K. Arthur, M. C. Whitton, R. Bastos, A. Steed, M. Slater, and F. P. Brooks Jr, “Walking> walking-in-place> flying, in virtual environments,” in Proceed- ings of the 26th annual conference on Computer graphics and interactive techniques, pp. 359–364, ACM Press/Addison-Wesley Publishing Co., 1999. [5] S. S. Chance, F. Gaunet, A. C. Beall, and J. M. Loomis, “Locomotion mode affects the updating of objects encountered during travel: The contribution of vestibular and proprioceptive inputs to path integration,” Presence, vol. 7, no. 2, pp. 168–178, 1998. [6] R. A. Ruddle and S. Lessels, “The benefits of using a walking interface to navi- gate virtual environments,” ACM Transactions on Computer-Human Interaction (TOCHI), vol. 16, no. 1, p. 5, 2009. [7] E. A. Suma, S. L. Finkelstein, S. Clark, P. Goolkasian, and L. F. Hodges, “Effects of travel technique and gender on a divided attention task in a virtual environment,” in 2010 IEEE Symposium on 3D User Interfaces (3DUI), pp. 27–34, IEEE, 2010. [8] C. A. Zanbaka, B. C. Lok, S. V. Babu, A. C. Ulinski, and L. F. Hodges, “Comparison of path visualizations and cognitive measures relative to travel technique in a virtual environment,” IEEE Transactions on Visualization and Computer Graphics, vol. 11, no. 6, pp. 694–705, 2005. [9] S. Razzaque, Z. Kohn, and M. C. Whitton, “Redirected walking,” Eurographics, 2001. 83 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . [10] K. Vasylevska, H. Kaufmann, M. Bolas, and E. A. Suma, “Flexible spaces: Dynamic layout generation for infinite walking in virtual environments,” in 2013 IEEE Symposium on 3D User Interfaces (3DUI), pp. 39–42, IEEE, 2013. [11] K. Vasylevska and H. Kaufmann, “Towards efficient spatial compression in self- overlapping virtual environments,” in 2017 IEEE Symposium on 3D User Interfaces (3DUI), pp. 12–21, IEEE, 2017. [12] T. Cakmak and H. Hager, “Cyberith virtualizer: a locomotion device for virtual reality,” in ACM SIGGRAPH 2014 Emerging Technologies, pp. 1–1, 2014. [13] M. Marchal, A. Lecuyer, G. Cirio, L. Bonnet, and M. Emily, “Walking up and down in immersive virtual worlds: Novel interactive techniques based on visual feedback,” in Proceedings of the 2010 IEEE Symposium on 3D User Interfaces, 3DUI ’10, (Washington, DC, USA), pp. 19–26, IEEE Computer Society, 2010. [14] R. Nagao, K. Matsumoto, T. Narumi, T. Tanikawa, and M. Hirose, “Infinite stairs: simulating stairs in virtual reality based on visuo-haptic interaction,” in ACM SIGGRAPH 2017 Emerging Technologies, p. 14, ACM, 2017. [15] R. Nagao, K. Matsumoto, T. Narumi, T. Tanikawa, and M. Hirose, “Walking up virtual stairs based on visuo-haptic interaction,” in ACM SIGGRAPH 2017 Posters, p. 36, ACM, 2017. [16] R. Nagao, K. Matsumoto, T. Narumi, T. Tanikawa, and M. Hirose, “Ascending and descending in virtual reality: Simple and safe system using passive haptics,” IEEE transactions on visualization and computer graphics, vol. 24, no. 4, pp. 1584–1593, 2018. [17] H. Iwata, H. Yano, and F. Nakaizumi, “Gait master: A versatile locomotion interface for uneven virtual terrain,” in Proceedings IEEE Virtual Reality 2001, pp. 131–137, IEEE, 2001. [18] J. Yoon and J. Ryu, “A novel locomotion interface with two 6-dof parallel manip- ulators that allows human walking on various virtual terrains,” The International Journal of Robotics Research, vol. 25, no. 7, pp. 689–708, 2006. [19] H. Q. Dinh, N. Walker, L. F. Hodges, C. Song, and A. Kobayashi, “Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments,” in Proceedings IEEE Virtual Reality (Cat. No. 99CB36316), pp. 222–228, IEEE, 1999. [20] J. M. Kinsella-Shaw, B. Shaw, and M. Turvey, “Perceiving’walk-on-able’slopes,” Ecological Psychology, vol. 4, no. 4, pp. 223–239, 1992. [21] D. R. Proffitt, M. Bhalla, R. Gossweiler, and J. Midgett, “Perceiving geographical slant,” Psychonomic bulletin & review, vol. 2, no. 4, pp. 409–428, 1995. 84 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . [22] D. R. Proffitt, S. H. Creem, and W. D. Zosh, “Seeing mountains in mole hills: Geographical-slant perception,” Psychological Science, vol. 12, no. 5, pp. 418–423, 2001. [23] M. Bhalla and D. R. Proffitt, “Visual–motor recalibration in geographical slant per- ception.,” Journal of experimental psychology: Human perception and performance, vol. 25, no. 4, p. 1076, 1999. [24] C. R. Riener, J. K. Stefanucci, D. R. Proffitt, and G. Clore, “An effect of mood on the perception of geographical slant,” Cognition and Emotion, vol. 25, no. 1, pp. 174–182, 2011. [25] J. K. Stefanucci, D. R. Proffitt, G. L. Clore, and N. Parekh, “Skating down a steeper slope: Fear influences the perception of geographical slant,” Perception, vol. 37, no. 2, pp. 321–323, 2008. [26] S. Schnall, K. D. Harber, J. K. Stefanucci, and D. R. Proffitt, “Social support and the perception of geographical slant,” Journal of experimental social psychology, vol. 44, no. 5, pp. 1246–1255, 2008. [27] D. R. Proffitt, “Embodied perception and the economy of action,” Perspectives on psychological science, vol. 1, no. 2, pp. 110–122, 2006. [28] C. Firestone, “How “paternalistic” is spatial perception? why wearing a heavy back- pack doesn’t—and couldn’t—make hills look steeper,” Perspectives on Psychological Science, vol. 8, no. 4, pp. 455–473, 2013. [29] J. K. Stefanucci and D. R. Proffitt, “The roles of altitude and fear in the perception of height.,” Journal of Experimental Psychology: Human Perception and Performance, vol. 35, no. 2, p. 424, 2009. [30] E. M. Clerkin, M. W. Cody, J. K. Stefanucci, D. R. Proffitt, and B. A. Teachman, “Imagery and fear influence height perception,” Journal of anxiety disorders, vol. 23, no. 3, pp. 381–386, 2009. [31] J. K. Stefanucci, K. T. Gagnon, and D. A. Lessard, “Follow your heart: Emotion adaptively influences perception,” Social and personality psychology compass, vol. 5, no. 6, pp. 296–308, 2011. [32] M. J. Schuemie, P. Van Der Straaten, M. Krijn, and C. A. Van Der Mast, “Research on presence in virtual reality: A survey,” CyberPsychology & Behavior, vol. 4, no. 2, pp. 183–201, 2001. [33] J. J. Gibson, The ecological approach to visual perception: classic edition. Psychology Press, 2014. [34] P. Zahorik and R. L. Jenison, “Presence as being-in-the-world,” Presence, vol. 7, no. 1, pp. 78–89, 1998. 85 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . [35] B. G. Witmer and M. J. Singer, “Measuring presence in virtual environments: A presence questionnaire,” Presence, vol. 7, no. 3, pp. 225–240, 1998. [36] K. M. Lee, “Presence, explicated,” Communication theory, vol. 14, no. 1, pp. 27–50, 2004. [37] K. Kilteni, R. Groten, and M. Slater, “The sense of embodiment in virtual reality,” Presence: Teleoperators and Virtual Environments, vol. 21, no. 4, pp. 373–387, 2012. [38] A. Alzayat, M. Hancock, and M. A. Nacenta, “Quantitative measurement of tool embodiment for virtual reality input alternatives,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–11, 2019. [39] M. Botvinick and J. Cohen, “Rubber hands ‘feel’touch that eyes see,” Nature, vol. 391, no. 6669, pp. 756–756, 1998. [40] B. Lenggenhager, M. Mouthon, and O. Blanke, “Spatial aspects of bodily self- consciousness,” Consciousness and cognition, vol. 18, no. 1, pp. 110–117, 2009. [41] M. O. Ernst and M. S. Banks, “Humans integrate visual and haptic information in a statistically optimal fashion,” Nature, vol. 415, no. 6870, p. 429, 2002. [42] M. O. Ernst, M. S. Banks, and H. H. Bülthoff, “Touch can change visual slant perception,” Nature neuroscience, vol. 3, no. 1, p. 69, 2000. [43] J. R. Lackner and P. DiZio, “Vestibular, proprioceptive, and haptic contributions to spatial orientation,” Annu. Rev. Psychol., vol. 56, pp. 115–147, 2005. [44] J. R. Lackner and A. Graybiel, “Elicitation of vestibular side effects by regional vibration of the head,” tech. rep., NAVAL AEROSPACE MEDICAL RESEARCH LAB PENSACOLA FL, 1974. [45] G. M. Goodwin, D. I. McCloskey, and P. B. Matthews, “Proprioceptive illusions induced by muscle vibration: contribution by muscle spindles to perception?,” Science, vol. 175, no. 4028, pp. 1382–1384, 1972. [46] J. Lackner and M. Levine, “Changes in apparent body orientation and sensory localization induced by vibration of postural muscles: vibratory myesthetic illusions.,” Aviation, space, and environmental medicine, 1979. [47] J. R. Lackner, “Some proprioceptive influences on the perceptual representation of body shape and orientation,” Brain, vol. 111, no. 2, pp. 281–297, 1988. [48] K. Hagbarth, “Motor effects of vibratory muscle stimuli in man,” in Noble Symposium I. Muscular afferents and motor control, pp. 177–186, Almqvist & Wiksell, 1966. [49] N. P. M. Todd, S. M. Rosengren, and J. G. Colebatch, “Tuning and sensitivity of the human vestibular system to low-frequency vibration,” Neuroscience letters, vol. 444, no. 1, pp. 36–41, 2008. 86 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . [50] M. Marchal, G. Cirio, Y. Visell, F. Fontana, S. Serafin, J. Cooperstock, and A. Lécuyer, “Multimodal rendering of walking over virtual grounds,” in Human Walking in Virtual Environments, pp. 263–295, Springer, 2013. [51] R. Kennedy, J. Drexler, D. Compton, K. Stanney, D. Lanham, and D. Harm, “Configural scoring of simulator sickness, cybersickness and space adaptation syn- drome: similarities and differences,” Virtual and Adaptive Environments Applications Implications and Human Performance Issues, pp. 247–278, 2003. [52] S. Weech, S. Kenny, and M. Barnett-Cowan, “Presence and cybersickness in virtual reality are negatively related: a review,” Frontiers in psychology, vol. 10, p. 158, 2019. [53] L. Rebenitsch and C. Owen, “Review on cybersickness in applications and visual displays,” Virtual Reality, vol. 20, no. 2, pp. 101–125, 2016. [54] S. Weech, J. Moon, and N. F. Troje, “Influence of bone-conducted vibration on simulator sickness in virtual reality,” PloS one, vol. 13, no. 3, p. e0194137, 2018. [55] S. Weech, T. Wall, and M. Barnett-Cowan, “Reduction of cybersickness during and immediately following noisy galvanic vestibular stimulation,” Experimental Brain Research, pp. 1–11, 2020. [56] Y.-H. Peng, C. Yu, S.-H. Liu, C.-W. Wang, P. Taele, N.-H. Yu, and M. Y. Chen, “Walkingvibe: Reducing virtual reality sickness and improving realism while walking in vr using unobtrusive head-mounted vibrotactile feedback,” 2020. [57] J. Marshall, S. Benford, R. Byrne, and P. Tennent, “Sensory alignment in immersive entertainment,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, p. 700, ACM, 2019. [58] E. A. Suma, S. Clark, D. Krum, S. Finkelstein, M. Bolas, and Z. Warte, “Leveraging change blindness for redirection in virtual environments,” in 2011 IEEE Virtual Reality Conference, pp. 159–166, IEEE, 2011. [59] E. A. Suma, Z. Lipps, S. Finkelstein, D. M. Krum, and M. Bolas, “Impossible spaces: Maximizing natural walking in virtual environments with self-overlapping architecture,” IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 4, pp. 555–564, 2012. [60] F. Steinicke, G. Bruder, J. Jerald, H. Frenz, and M. Lappe, “Analyses of human sensitivity to redirected walking,” in Proceedings of the 2008 ACM symposium on Virtual reality software and technology, pp. 149–156, ACM, 2008. [61] F. Steinicke, G. Bruder, J. Jerald, H. Frenz, and M. Lappe, “Estimation of detection thresholds for redirected walking techniques,” IEEE transactions on visualization and computer graphics, vol. 16, no. 1, pp. 17–27, 2010. 87 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . [62] T. Grechkin, J. Thomas, M. Azmandian, M. Bolas, and E. Suma, “Revisiting detection thresholds for redirected walking: Combining translation and curvature gains,” in Proceedings of the ACM Symposium on Applied Perception, pp. 113–120, ACM, 2016. [63] E. A. Suma, G. Bruder, F. Steinicke, D. M. Krum, and M. Bolas, “A taxonomy for deploying redirection techniques in immersive virtual environments,” in 2012 IEEE Virtual Reality Workshops (VRW), pp. 43–46, IEEE, 2012. [64] M. Azmandian, T. Grechkin, M. Bolas, and E. Suma, “The redirected walking toolkit: a unified development platform for exploring large virtual environments,” in 2016 IEEE 2nd Workshop on Everyday Virtual Reality (WEVR), pp. 9–14, IEEE, 2016. [65] G. Bruder, F. Steinicke, and P. Wieland, “Self-motion illusions in immersive virtual reality environments,” in 2011 IEEE Virtual Reality Conference, pp. 39–46, IEEE, 2011. [66] G. Bruder, F. Steinicke, and K. H. Hinrichs, “Arch-explore: A natural user interface for immersive architectural walkthroughs,” in 2009 IEEE Symposium on 3D User Interfaces, pp. 75–82, IEEE, 2009. [67] M. Slater, M. Usoh, and A. Steed, “Steps and ladders in virtual reality,” in Virtual Reality Software And Technology, pp. 45–54, World Scientific, 1994. [68] C. Lai, R. P. McMahan, and J. Hall, “March-and-reach: A realistic ladder climbing technique,” in 2015 IEEE Symposium on 3D User Interfaces (3DUI), pp. 15–18, IEEE, 2015. [69] K. Matsumoto, T. Narumi, T. Tanikawa, and M. Hirose, “Walking uphill and downhill: redirected walking in the vertical direction,” in ACM SIGGRAPH 2017 Posters, p. 37, ACM, 2017. [70] N. S. Asjad, H. Adams, R. Paris, and B. Bodenheimer, “Perception of height in virtual reality: a study of climbing stairs,” in Proceedings of the 15th ACM Symposium on Applied Perception, p. 4, ACM, 2018. [71] K. Vasylevska, H. Kaufmann, and V. Khrystyna, “Influence of vertical navigation metaphors on presence,” in Proceedings of 15th International Conference on Presence (ISPR 2014); Vienna, Austria, pp. 17–19, 2014. [72] U. Technologies, “Unity game engine.” https://unity.com/. [Online; accessed: 20.01.2020]. [73] V. Corporation, “Steamvr.” https://store.steampowered.com/app/ 250820/SteamVR/. [Online; accessed: 20.01.2020]. 88 D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . D ie a pp ro bi er te g ed ru ck te O rig in al ve rs io n di es er D ip lo m ar be it is t a n de r T U W ie n B ib lio th ek v er fü gb ar . T he a pp ro ve d or ig in al v er si on o f t hi s th es is is a va ila bl e in p rin t a t T U W ie n B ib lio th ek . [74] V. Corporation, “Steamvr unity plugin.” https://assetstore.unity.com/ packages/tools/integration/steamvr-plugin-32647. [Online; ac- cessed: 10.04.2020]. [75] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal, “Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness,” The international journal of aviation psychology, vol. 3, no. 3, pp. 203–220, 1993. 89