Effects of Multisensory
Stimulation in Virtual Reality
DIPLOMARBEIT
zur Erlangung des akademischen Grades
Diplom-Ingenieur
im Rahmen des Studiums
Media and Human-Centered Computing
eingereicht von
B.Sc. Sebastian Reissmüller
Matrikelnummer 11945414
an der Fakultät für Informatik
der Technischen Universität Wien
Betreuung: Ao.Univ.Prof. Mag. Dr. Horst Eidenberger
Wien, 22. November 2024
Sebastian Reissmüller Horst Eidenberger
Technische Universität Wien
A-1040 Wien Karlsplatz 13 Tel. +43-1-58801-0 www.tuwien.at

Effects of Multisensory
Stimulation in Virtual Reality
DIPLOMA THESIS
submitted in partial fulfillment of the requirements for the degree of
Diplom-Ingenieur
in
Media and Human-Centered Computing
by
B.Sc. Sebastian Reissmüller
Registration Number 11945414
to the Faculty of Informatics
at the TU Wien
Advisor: Ao.Univ.Prof. Mag. Dr. Horst Eidenberger
Vienna, November 22, 2024
Sebastian Reissmüller Horst Eidenberger
Technische Universität Wien
A-1040 Wien Karlsplatz 13 Tel. +43-1-58801-0 www.tuwien.at

Erklärung zur Verfassung der
Arbeit
B.Sc. Sebastian Reissmüller
Hiermit erkläre ich, dass ich diese Arbeit selbständig verfasst habe, dass ich die verwen-
deten Quellen und Hilfsmittel vollständig angegeben habe und dass ich die Stellen der
Arbeit – einschließlich Tabellen, Karten und Abbildungen –, die anderen Werken oder
dem Internet im Wortlaut oder dem Sinn nach entnommen sind, auf jeden Fall unter
Angabe der Quelle als Entlehnung kenntlich gemacht habe.
Ich erkläre weiters, dass ich mich generativer KI-Tools lediglich als Hilfsmittel bedient
habe und in der vorliegenden Arbeit mein gestalterischer Einfluss überwiegt. Im Anhang
„Übersicht verwendeter Hilfsmittel“ habe ich alle generativen KI-Tools gelistet, die
verwendet wurden, und angegeben, wo und wie sie verwendet wurden. Für Textpassagen,
die ohne substantielle Änderungen übernommen wurden, haben ich jeweils die von
mir formulierten Eingaben (Prompts) und die verwendete IT- Anwendung mit ihrem
Produktnamen und Versionsnummer/Datum angegeben.
Wien, 22. November 2024
Sebastian Reissmüller
v

Danksagung
Ich möchte mich bei meinem Betreuer Prof. Dr. Horst Eidenberger für die Flexbilität, den
Optimismus und die allgemeine Unterstützung bedanken. Desweiteren möchte ich mich
bei meinen Kolleg:innen vom CSD für ihr Verständnis und ihre Unterstützung bedanken.
Ich möchte auch die unglaubliche Arbeit von Forschern wie Lanier, Slater und Palmer
würdigen, die mit ihrer Vision den Grundstein für den Bereich der VR gelegt haben.
vii

Acknowledgements
I would like to express my sincere gratitude to my supervisor Prof. Dr. Horst Eidenberger
for his flexibility, optimism and complete support. Furthermore I would like to thank my
colleagues at the CSD for their understanding and support. I’d also like to acknowledge
the incredible work of researchers like Lanier, Slater and Palmer, whose vision laid the
groundwork in the field of VR.
ix

Kurzfassung
Die vorliegende Arbeit befasst sich mit den Auswirkungen der Integration von multi-
sensorischen Feedback, insbesondere der Kombination von 3D-Audio und haptischen
Elementen. Im Rahmen der Studie wurde der Einfluss auf die Kernmetriken ’immersion’,
’presence’ und ’motion sickness’ untersucht. Die vorliegende Untersuchung schließt eine
Lücke in der bestehenden Forschungsliteratur zum Thema Virtual Reality, indem sie die
Auswirkungen von synchronisiertem sensorischem Feedback auf die Benutzererfahrung
über den Bereich der visuellen Immersion hinaus untersucht. Zu diesem Zweck wurde das
VR-System ’Phantastic Sound Experience’ entwickelt. Das System verbindet 3D-Audio
mit taktilem Feedback, indem es eine virtuelle Umgebung mit einem eigens dafür an-
gefertigten Konstrukt kombiniert, welches audio-ausgelöste taktile Vibrationen erzeugt.
Im Rahmen der Evaluierung des Projekts wurden zwei Versionen des Systems getestet,
um eine Gegenüberstellung der Ergebnisse zu ermöglichen. Die Datenerhebung erfolgte
mittels standardisierter Fragebögen zur Erfassung der Präsenz- und Immersionserfahrung
sowie durch leitfadengestützte Nutzerinterviews. Die Implementierung multisensorischen
Feedbacks resultierte in einer leichten Intensivierung der ’immersion’ und ’presence’ der
Testpersonen. Die Teilnehmer:innen der Gruppe B, welche sowohl taktiles als auch akusti-
sches Feedback erhielten, berichteten über eine verstärkte körperliche Reaktion sowie eine
Verringerung der Wahrnehmung der realen Welt. Während bei einigen Messwerten keine
statistisch signifikanten Unterschiede festgestellt wurden, deutete qualitatives Feedback
auf ein stärkeres Gefühl der Zeitverzerrung und der ’presence’ bei denjenigen hin, welche
multisensorische Hinweise erhielten. Die Ergebnisse legen nahe, dass die Kombination
von 3D-Audio und haptischen Reizen in VR das Potenzial birgt, die Nutzerbindung
zu intensivieren. Dennoch sind weitere Forschungsarbeiten erforderlich, um die für die
Integration der sensorischen Reize eingesetzten Techniken zu verfeinern. Die vorliegende
Studie präsentiert Erkenntnisse zur effektiven sensorischen Synchronisation in VR und
identifiziert Forschungsfelder für künftige Untersuchungen. Dazu zählen die Optimierung
von Kalibrierungsmethoden für haptisches Feedback sowie die Analyse individueller
sensorischer Wahrnehmung. Die Resultate dieser Studie bilden die Basis für weitere
Projekte die multisensorische Stimulierung einsetzen, um ein umfassendes immersives
Nutzerlebnis ermöglichen.
xi

Abstract
This thesis investigates the impact of integrating multisensory feedback, particularly the
combination of 3D audio and haptic elements, on the enhancement of immersion and
presence in virtual reality environments. Addressing a significant gap in the existing VR
research literature by examining the impact of synchronized sensory feedback on user
experience, extending beyond the domain of visual immersion. To investigate this, the
"Phantastic Sound Experience" VR system was developed. The system is implemented
to combine 3D audio and tactile feedback by combining a virtual environment with
a custom built physical device, that provides audio-triggered tactile vibrations. The
participants were exposed to two conditions: one incorporating multisensory feedback
and another comprising solely the visual environment. Quantitative and qualitative
data were gathered through the administration of standardized presence and immersion
questionnaires, complemented by user interviews. The findings indicate that the provision
of multisensory feedback has the effect of enhancing user immersion and presence. In
particular, those in Group B (who received tactile and audio feedback) reported increased
physical involvement and a reduction in their awareness of the real world. While no
statistically significant differences were observed in some metrics, qualitative feedback
indicated a stronger sense of time distortion and "being there" among those receiving
multisensory cues. These findings imply that the combination of 3D audio and haptics
in VR has the potential to enhance user engagement. However, further research is
required to refine the techniques employed for the integration of sensory stimuli. This
study provides insights into effective VR sensory synchronization and identifies areas for
future work, such as the optimization of calibration methods for haptic feedback and
the examination of individual sensory processing differences. These findings establish a
foundation for nuanced sensory synchronization in VR, advancing design strategies that
foster fully immersive user experiences.
xiii

Contents
Kurzfassung xi
Abstract xiii
Contents xv
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem statement and aim of work . . . . . . . . . . . . . . . . . . . 2
1.3 Methodological approach . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Thesis structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Literature review 5
2.1 Overview of Virtual Reality . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 History and present of VR . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Key factors for evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Sensory feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.5 Design principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.6 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3 Design and concept 25
3.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 Design approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.3 Conception of the VR experience . . . . . . . . . . . . . . . . . . . . . 31
3.4 Scientific process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4 Implementation 39
4.1 Virtual environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.2 Audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4.3 Physical setup and tactile feedback . . . . . . . . . . . . . . . . . . . . 44
4.4 Implementation of the experience in Unity . . . . . . . . . . . . . . . . 48
4.5 User tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4.6 Improved system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
xv
5 Evaluation 57
5.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.2 Evaluation methodology . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.4 Analysis and interpretation . . . . . . . . . . . . . . . . . . . . . . . . 67
5.5 Summary of findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
6 Conclusion and future work 75
6.1 Areas for improvement . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
6.2 Implications for future work . . . . . . . . . . . . . . . . . . . . . . . . 76
6.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Appendices 77
A Questionnaires 79
List of Figures 83
Bibliography 85
CHAPTER 1
Introduction
1.1 Motivation
Virtual Reality (VR) technology has developed rapidly from a concept initially situated
in the realms of science fiction to a medium with a range of applications across various
domains, including entertainment, education, and healthcare. The potential of VR to
create immersive and engaging experiences has led to considerable developments in
hardware, particularly in the area of high-resolution and low-latency head-mounted
displays. The author thinks that despite these advances in visual technology, the pursuit
of a fully immersive experience has often overlooked the critical role of other sensory
modalities, such as spatial audio and tactile feedback. Despite these advancements,
current VR technology also faces challenges such as high costs, technical limitations like
latency and resolution issues, and potential side effects such as motion sickness. Future
developments aim to address these limitations and expand VR’s accessibility, potentially
making it as ubiquitous as personal computers or smartphones in daily life [32].
Recent research has highlighted the significance of these non-visual sensory inputs in
developing a more comprehensive and immersive experience. A systematic review by
Melo et al. [53] indicates that the incorporation of multisensory stimuli, such as auditory
and haptic feedback, substantially enhances user engagement and the sense of presence in
virtual environments. Their findings suggest that combining multiple sensory modalities
can create a more realistic and immersive virtual reality experience, improving user
interaction and overall satisfaction. This underscores the importance of integrating
various sensory inputs to simulate real-world experiences more effectively.
The integration of spatially accurate audio cues, achieved through techniques such as
binaural audio, has been demonstrated to significantly contribute to the development
of spatial awareness and presence within virtual environments. Such auditory cues not
only enhance the overall immersive experience but also provide crucial information for
1
1. Introduction
orientation, proximity, and environmental context [88]. Furthermore, the incorporation of
tactile feedback mechanisms, such as haptic devices and vibrations, introduces a physical
dimension to virtual experiences, enabling users to perceive interactions, textures, and
environmental nuances. This tactile interaction serves to enhance the realism and
engagement of virtual reality environments, thereby blurring the line between the virtual
and physical worlds.
Although significant advances have been made in the visual field of VR systems, the po-
tential of auditory and tactile feedback remains under-researched. The current generation
of VR technologies tends to prioritize visual stimuli, with an emphasis on high-resolution
displays and advanced graphics. While these elements are crucial, they do not fully
leverage the potential of multisensory input to achieve a truly immersive experience [54].
These insights highlight the importance of investigating the role of these additional sensory
modalities in user immersion and presence, particularly in applications that require high
levels of engagement, such as psychological and medical interventions. The importance
of presence-inducing media for mental health applications has been demonstrated by
research, with significant influence on therapeutic outcomes identified [79].
This thesis seeks to address this gap in research by investigating the impact of auditory
and tactile feedback in virtual environments. By focusing on the integration of these
non-visual sensory inputs, this study aims to provide new insights into enhancing user
immersion, presence, and emotional engagement in VR, contributing to the development
of more effective and immersive virtual experiences.
1.2 Problem statement and aim of work
The fundamental research conducted by Slater and Wilbur [68] established the significance
of sensory stimuli in enhancing the sense of immersion and presence within virtual reality
environments. Subsequent studies by Cooper et al. [20] and Shaw et al. [66]. provided
further evidence of the significant impact of auditory and tactile stimuli on the user’s
sense of presence. Moreover, Bruun-Pedersen et al. [15] investigated the influence of
auditory and tactile feedback, yet could not substantiate a notable impact of the sensory
feedback. Given that the approach employed involved tasks and changing landscapes, the
author believes that the impact of sensory input could be tested with different parameters,
such as the user being passive in a more static environment. It is possible that these
parameters may influence the outcome of the experiment. While the impact of sensory
feedback in virtual reality is a topic of considerable research interest, relatively few studies
have explored the combined effects of spatial audio and tactile feedback, which could
enhance the immersive quality of the experience. Further investigation may therefore be
beneficial to assess the impact of these techniques on the overall experience of virtual
reality. This thesis seeks to address the aforementioned gap by conducting a systematic
investigation into the effects of integrating binaural audio and tactile feedback in virtual
reality environments. The following research questions will provide the framework for
this study:
2
1.3. Methodological approach
The primary objective of the authors research is to investigate the impact of integrating
binaural audio and tactile feedback in a virtual environment on the user experience. The
objective is to address the following research questions:
Research Question 1: How does the integration of 3D-audio and tactile stimuli
influence levels of immersion and presence in a VR environment?
Research Question 2: To what extent do these sensory inputs have an impact on
perceived motion sickness?
Research Question 3: What is the impact of sensory feedback on users’ emotional
states and the perceived pleasantness of the VR experiences?
The primary objective of this research is to address these questions, which serve to
establish the fundamental premise that multisensory feedback can positively influence
the key performance indicators for virtual reality experiences.
1.3 Methodological approach
In order to evaluate the impact of sensory feedback in virtual reality, a structured
methodology was adopted. This methodology is focused on the creation of logically
related physical and virtual systems, the iteration of user tests, and the comprehensive
evaluation through participant studies. The following presents a comprehensive overview
of the methodology employed, from the initial development of prototypes to the final
implementation and the subsequent study of participants. The methodology places
particular emphasis on the enhancement and assessment of sensory feedback mechanisms.
Literature review: The initial stage of the methodology involved a comprehensive literature
review, which examined existing research and studies related to the effects of binaural
audio and sensory feedback in virtual reality. This review provided a robust foundation
by summarizing and integrating current knowledge in the field. Given the recent surge in
VR research across various disciplines, the literature review highlighted recent studies
that offer valuable insights and a frame of reference for this project.
Development of a prototype: Once the literature review was complete, the next phase
of the project was to create a prototype for use with virtual reality technology that
incorporated binaural audio and sensory feedback. This practical implementation allowed
for hands-on experimentation, thereby enabling the assessment of the feasibility and
effectiveness of the proposed technologies.
Usability testing: Prior to the final implementation, the prototype was evaluated through
a usability test. This phase proved instrumental in enabling the system to be refined in
accordance with the feedback provided by users. A small number of participants tested
the prototype, providing feedback regarding the virtual world, sound and overall appeal.
The iterative process of testing and refinement ensured that the prototype was adequately
prepared for the subsequent user-based evaluation.
3
1. Introduction
Evaluation and analysis: The final phase of the study involved a user-based evaluation, in
which participants engaged with the VR prototype in a controlled study. The participants
were divided into two groups and completed questionnaires designed to measure their
experiences with spatial audio and sensory feedback. The questionnaire was based on
insights from previous research, including the Igroup Presence Questionnaire and the
Immersion Tendencies Questionnaire. The evaluation provided valuable feedback and
observations in order to evaluate the effectiveness of the sensory feedback mechanisms
employed.
This structured methodology allowed for a thorough exploration and assessment of the
impact of sensory feedback in VR, contributing to the development of more immersive
and realistic virtual environments.
1.4 Thesis structure
The structure of the thesis is designed to provide a comprehensive examination of the
theoretical foundations, practical implementations, and an empirical evaluation.
The second chapter presents a review of the existing literature on virtual reality and
related work, offering a detailed explanation of the key concepts of VR. The chapter
opens with an overview of the application and historical developments as well as current
advancements of VR. Subsequently, the key dimensions including presence, immersion,
motion sickness, and emotions, are examined in detail. Subsequently, the chapter examines
the function of sensory feedback in VR, with a focus on visual, auditory, and tactile
feedback mechanisms.
The following chapter will present the design and concept of the virtual reality system.
This entails outlining the system’s requirements, the design approach, and the system
conception. This chapter will examine specific elements of the system, including the
virtual environment, spatial audio and tactile feedback, and will explain the methodology
employed in developing each component and integrating them into the system.
The following section will present an outline of the development process. It covers the
design and integration of the virtual world, including the creation of digital assets, the
implementation of the previously mentioned design in Unity, and the incorporation
of visuals and sound. Additionally, the construction and integration of the hardware
for tactile feedback are described, along with technical aspects such as the physical
component and integration of the vibration module.
The evaluation chapter provides an overview of the conduction and analysis of the user
study. This chapter reflects on the objectives discussed in the first chapter, evaluates the
effectiveness of the methodology employed, and discusses the implications of the findings.
In the last chapter the author reflects on the challenges of the project and identifies
potential of future developments.
4
CHAPTER 2
Literature review
In this chapter the key aspects of virtual reality will be examined, including its applications,
historical development, and current technological landscape. The first section provides
an overview of VR applications in sectors such as gaming, healthcare, robotics and
manufacturing. We then delve into the history and present of VR, exploring the milestones
that have shaped its development from the 1960s to the present day. The section is
followed by the analysis of the critical components of VR systems, including the hardware
and sensory feedback mechanisms that play a key role in creating immersive experiences.
Finally, the chapter concludes with an assessment of related work and key factors that
influence the effectiveness of VR, setting the stage for the research objectives of this
thesis.
2.1 Overview of Virtual Reality
Virtual Reality has evolved rapidly as a transformative medium, offering immersive
experiences beyond traditional interaction methods. Its applications cover a wide range
of fields such as entertainment, education and health care, providing users with unique
opportunities to interact with digital environments that were previously unimaginable.
This chapter focuses on the fundamental concepts of VR technology, its applications and
describes in detail the main components that make up the VR system.
2.1.1 VR applications
With a novel way to interact in a virtual space VR shows potential to branch into different
areas of application. Although the immersion of users in virtual environments is primarily
a visual experience, it also has spatial awareness-enhancing and interactive capabilities.
This allows users to interact with these virtual environments as if they were the real
thing, which has significant implications for how we understand the boundaries between
5
2. Literature review
reality and fiction. As noted by Hamad and Jia [32], "VR has the potential to be a highly
beneficial tool in a variety of applications and across a wide range of fields", including
education, healthcare, gaming and engineering. Furthermore, Javaid and Haleem [42]
emphasize the utility of VR across industries, describing it as "a promising technology
that is finding its place in a range of fields, including manufacturing, education, and
medical applications."
Gaming has been one of the earliest and most prominent sectors to adopt VR technology,
bringing a new level of interaction to digital entertainment. VR games allow players to
immerse themselves in virtual experiences where they can explore, manipulate objects and
control their surroundings through intuitive, natural movements. The level of engagement
offered by VR enhances the gaming experience, giving players a stronger emotional
connection to the gameplay and story. Platforms such as Oculus Rift, HTC Vive and
PlayStation VR have made VR gaming more accessible, with a wide range of titles from
high-action shooters to puzzle and exploration games. The high-fidelity graphics, spatial
audio, and advanced haptics work together to create an immersive environment that sets
VR gaming apart from traditional experiences. VR is making significant contributions to
the medical field, particularly in training, diagnosis and therapy. "VR allows medical
students and professionals to practice complex procedures in a risk-free, controlled
environment," as Hamad and Jia [32], explain. Surgical simulations, for example, provide
a realistic hands-on experience that helps practitioners hone their skills without the need
for real patients. Javaid and Haleem [42] elaborate on this, noting that VR "enhances
the precision and skills of medical professionals through detailed surgical simulations". In
therapeutic settings, VR is also used in exposure therapy for anxiety and post-traumatic
stress disorder (PTSD), allowing patients to confront fears in a controlled environment.
In physical rehabilitation, VR is used to help patients regain motor function through
interactive exercises that monitor progress and motivate engagement.
In robotics, VR serves as a powerful tool for training and control systems. VR creates
virtual representations of physical environments, allowing operators to remotely control
robots with precision while visualising real-time data in a simulated 3D environment.
This is particularly valuable in industries such as space exploration and defence, where
robots are used in dangerous or inaccessible environments. "VR enables more intuitive
human-robot interaction by allowing operators to manipulate virtual robots in real time,"
improving accuracy and reducing operational errors. Javaid and Haleem [42] also point
out that "virtual environments in robotics training allow for better control of complex
tasks, reducing errors and increasing safety".
The production and manufacturing sector is also benefiting from VR, which is being used to
"optimize workflows, improve training and enhance product design". VR simulations allow
engineers and managers to design and test production lines before physical implementation,
reducing costs and helping to identify inefficiencies early in the process. Hamad and Jia
[32] note that "VR facilitates collaborative design by allowing team members to interact
with virtual prototypes in real time" and make changes to product designs, even when
they are remotely located. VR-based training programmes also help workers familiarise
6
2.1. Overview of Virtual Reality
themselves with machinery and safety protocols in a virtual space, reducing on-the-job
risks and improving overall efficiency.
The successful application of VR in these industries relies on the seamless integration of
several key components that form the foundation of VR systems. In the next section,
these components will be explored and explained how they work together to create
immersive, interactive environments.
2.1.2 System components
A virtual reality system comprises essential key components that collectively facilitate the
creation of an immersive and interactive digital environment. At the core of this system
is the VR headset (C), which typically includes high-resolution stereoscopic displays
for each eye. These are designed to deliver a three-dimensional visual experience by
simulating depth perception. Such headsets are frequently furnished with accelerometers,
gyroscopes, and magnetometers, which enable the accurate tracking of head movements.
This allows for real-time adjustments to the user’s viewpoint within the virtual space,
which is vital for maintaining a coherent sense of presence and minimizing discomfort
such as motion sickness. A further essential component is the tracking system (A),
which captures the user’s spatial movements and orientation. This system may employ
external sensors or cameras, as exemplified by the HTC Vive system, or alternatively,
it may rely on internal tracking mechanisms embedded within the headset, as observed
in devices such as the Oculus Quest. The computational unit (E), which may take the
form of a high-performance computer, gaming console, or integrated processing unit, is
responsible for rendering the virtual environment, processing real-time user inputs, and
ensuring seamless operation in order to support a fluid and responsive VR experience.
The computational device must be connected to the HMD via suiting cables, mostly
Display-link and USB-C (D). Furthermore, input devices (B), such as handheld controllers,
haptic gloves, or motion sensors, facilitate user interaction with and manipulation of
virtual objects, thereby enhancing the interactivity and engagement of the VR experience.
The aforementioned components collectively constitute a unified system that serves as
the foundation for the functionality of VR [87]. In addition, a working area (F), which
encompasses the spatial boundaries of the virtual world, must be defined. This system
enables users to fully immerse themselves in digitally constructed environments and
interact with them in a manner that is intuitive and natural.
The core of VR technology is built on three interconnected concepts: immersion, presence,
and interaction. The term "immersion" is used to describe the capacity of a virtual
reality system to encourage the user to engage with sensory information in a way that is
perceived as being real. Presence is the psychological phenomenon in which users perceive
themselves to exist in a virtual environment rather than in a physical one. Interactivity
measures the extent to which users can influence and react to the virtual world, thus
improving the sense of realism and engagement [39]. These elements are fundamental
to the ability of VR to create rich and interactive experiences in various fields, from
games to therapeutic applications. Understanding these core concepts is vital not only
7
2. Literature review
Figure 2.1: Key components of a VR setup [71]
to understand the capabilities and limitations of current VR technologies, but also to
identify potential areas of innovation and improvement. After introducing the basic
concepts of VR, the following sections will delve further into the historical evolution of
this technology, its current state and the critical components that form a VR system.
8
2.2. History and present of VR
Understanding the current technological capabilities of VR requires a look at the path it
has followed. In the following section, the history of VR will be traced from its early
conceptual roots to the advanced systems of today, providing context for its present-day
applications
2.2 History and present of VR
The development of virtual reality technology is marked by significant milestones, which
have gradually shaped its current form over the decades. The concept of VR goes back
to the 1960s, when Sutherland [69] introduced the concept of Ultimate Display in 1965,
whose concept was a virtual world in which users could interact with a head-mounted
screen (HMD) This conceptual framework laid the foundations for the development of
early VR systems. In the late 1960s, Sutherland and his student Bob Sproull created the
first HMD, known as Damocles’ sword. Although primitive in current standards, this
device was a pioneer of virtual experience immersion [70].
Figure 2.2: The data glove by Lanier [81]
In the 1980s the introduction of VPL Research’s Data Glove, founded by Jaron Lanier,
made further advances. Lanier is recognized for popularizing the term "virtual reality"
and is considered one of the pioneers in this field. His vision and work influenced the
9
2. Literature review
direction of VR technology. The data glove ( shown in figure 2.2 ) allows users to interact
with virtual objects with hand gestures, which paved the way for more interactive VR
systems [6]. Lanier’s contributions go beyond hardware, exploring the philosophical and
social impacts of VR, and highlighting the potential of VR to expand human experience
and creativity [48].
In the 1990s, VR became popular in commercial systems such as virtuality arcade
machines and Sega VR headsets. However, these initial attempts were limited by
technology constraints at the time, such as low-resolution displays, limited processing
power, and high costs, which prevented widespread adoption. Despite these limitations,
major research was conducted in the 1990s on the potential application of VR beyond
gaming, including its use in training simulations, medical therapy and architectural
design [13].
The following period has witnessed significant advances that have driven VR from a niche
technology to a more common application. In the early 2000s, VR research focused on
improving user interaction and realism. Developments in motion tracking technology,
such as the introduction of optical tracking systems, have enabled more accurate and
receptive VR experiences [30]. These advances have helped to improve the user’s sense
of presence and interaction in the virtual environment.
In the mid-2000s, immersive VR platforms such as CAVE (Cave Automatic Virtual
Environment) systems emerged, providing users with a room-sized immersive experience
[21]. These systems are particularly popular in academic and industrial research and are
used for tasks ranging from data visualization to architectural architecture. The CAVE
system is an excellent example of these qualities, providing an immersive, high-resolution
display that significantly enhances the user’s sense of presence. The technology achieves
this by creating a multi-screen environment that surrounds the user, providing a more
immersive experience than traditional head-mounted displays (HMDs) [68]
Unlike HMDs, which have the potential to restrict the field of view and cause disorientation,
the expansive surrounding screens of the CAVE provide a panoramic field of view. This
configuration not only enhances spatial awareness, but also facilitates the interaction of
multiple users in a shared virtual environment, optimizing its suitability for collaborative
tasks. Such systems have been shown to be particularly effective in applications that
require detailed spatial understanding, including architectural visualization, complex data
analysis, and training simulations, where a high degree of presence can enhance learning
outcomes and facilitate the transfer of skills to real-world scenarios [46]. The immersive
experience provided by CAVE systems can be further enhanced by integrating multimodal
sensory feedback. Combining visual stimuli with auditory and haptic feedback has been
shown to significantly improve user performance and reduce errors in virtual reality
environments. This multimodal approach ensures that the user’s sensory input is more
closely aligned with real-world experiences, thereby deepening immersion and increasing
the sense of presence.
In 2012, the launch of the Oculus Rift marked a turning point in VR history. Oculus,
10
2.2. History and present of VR
Figure 2.3: The CAVE System Setup [82]
founded by Palmer Luckey, has launched VR into the consumer market with affordable,
high-quality HMDs that offer a wide viewing range, low latency and high resolution
displays [50]. The success of the Oculus Rift sparked a wave of innovation and led to the
development of other popular VR headsets, such as the HTC Vive and PlayStation VR,
which democratized access to immersive experiences.
Between 2015 and 2020, VR technology experienced rapid advances in hardware and
software. The introduction of the 6DoF controller (Six Degrees of Freedom) allowed users
to move freely and interact more naturally with the virtual environment [5]. In addition,
the development of graphic rendering, such as real-time radiation tracking, has made the
environment more realistic and visually stunning [78]. The integration of haptic feedback
and spatial audio further enhances immersive experience, allowing users to feel and hear
virtual worlds as real.
VR technology has made significant progress between 2020 and 2024, particularly in the
field of hardware and software development. One of the most remarkable developments
was the enhancement of the independent VR headsets, which no longer required connection
to a powerful computer, making VR more accessible and user-friendly. Devices such as
Oculus Quest 2 introduced higher resolution screens, improved refresh rates and advanced
internal tracking, significantly improving user experience [80]. These improvements in
hardware have been supplemented by innovations in rendering technologies, such as
foveated rendering, which reduces computational load by adjusting the resolution based
on the user’s orientation, thus improving performance and immersion [44].
VR applications have experienced a significant growth in several areas beyond entertain-
ment, including education, health care and remote collaboration. In the medical sector,
11
2. Literature review
VR has increasingly been used for therapeutic interventions, especially in the treatment
of mental illnesses such as PTSD and anxiety disorders. Studies have shown that VR
therapy can offer immersive exposure therapy, allowing patients to confront their fears in
a controlled virtual environment, leading to significant improvements in clinical results
[47].
The COVID-19 pandemic further accelerated the adoption of VR for remote collabo-
ration, with organizations and educational institutions using VR platforms to create
virtual meeting spaces that replicate closely real interactions and effectively address the
limitations of traditional video conferences tools. During this period, the design of virtual
classrooms in virtual environments became a decisive innovation to improve access and
quality to remote education. This innovation enabled interactive and immersive learning
experiences, offering students the opportunity to participate in educational content that
traditional online platforms cannot [51]. The integration of artificial intelligence into
these VR environments further enhanced the interaction and adaptability of virtual
scenarios, expanding the scope and effectiveness of VR applications in various sectors.
Major players in the industry include Facebook (Meta), which acquired Oculus and has
invested heavily in the development of VR, HTC, known for its Vive headsets, and Sony
with its PlayStation VR. In addition, companies such as Valve, Samsung, and Microsoft
are making significant contributions to the VR ecosystem [36]. Market trends show a
continuous expansion of VR applications that focus on improving user experience through
higher resolution displays, better ergonomics, and more intuitive interactions. The
integration of augmented reality and mixed reality technologies is expected to stimulate
innovation and lead to a seamless and immersive experience.
The release of the HTC Vive Pro 2 in 2021 marked a new benchmark in VR with its 5K
resolution display, 120-degree field of view, and 120Hz refresh rate, offering one of the
most immersive experiences available to consumers [38]. Similarly, Apple’s introduction of
the Vision Pro in 2023 represented a major leap in mixed reality, combining VR and AR
capabilities in a sleek, all-in-one headset. The Vision Pro features advanced eye-tracking,
spatial audio, and the ability to interact seamlessly with digital content in the real world,
setting new standards for immersive technology [7]. Beyond hardware, the integration
of artificial intelligence and machine learning within VR environments has significantly
enhanced user interactivity and content generation, making virtual experiences more
dynamic and responsive. These advancements underscore the rapid evolution of VR
technology, positioning it as a crucial component of future digital experiences across
various industries.
In summary, the current state of VR technology reflects a mature and rapidly evolving
field. With ongoing advancements in hardware and software, VR is set to play an
increasingly integral role across diverse industries, offering transformative opportunities
for immersive and interactive experiences.
12
2.3. Key factors for evaluation
(a) HTC Vive Pro 2 kit [73] (b) Apple Vision Pro [8]
Figure 2.4: Modern VR HMDs
2.3 Key factors for evaluation
The efficacy of virtual reality as an immersive technology is dependent upon a number
of critical factors that, when considered collectively, serve to enhance the user experi-
ence. These key factors include presence, immersion, motion sickness, and emotions.
Understanding and optimizing these elements is essential for creating compelling VR
environments.
2.3.1 Presence
Presence is a fundamental aspect of VR and is often referred to as the “sense of being
there” [28]. This is a psychological state where users feel like they are really in the
virtual environment rather than in their physical environment. This feeling of presence
is important for the effectiveness of VR experiences because it directly influences user
engagement and satisfaction. Several factors contribute to the presence of VR. For
example, spatial audio plays an important role in providing auditory signals that align
with the visual environment, thus increasing the realism of the experience [29]. Haptic
feedback, which includes tactile sensations transmitted through devices such as haptic
gloves and vests, also strengthens the presence by allowing users to feel interactions
within the virtual world. In addition, high-resolution displays and accurate movement
tracking are crucial to minimizing visual and latency issues and ensure that user actions
are accurately reflected in the VR environment. Recent research has further explored
the relationship between sensory cues and presence in VR. The study by Cooper et al.
[20] demonstrated that the integration of substitute multisensory feedback, such as
auditory and tactile cues, can significantly enhance both task performance and the sense
of presence in VR environments. The findings suggest that even when sensory cues
are not fully realistic, their presence can still improve users’ immersive experience by
providing additional information that enhances interaction and involvement. This implies
that the strategic use of sensory feedback, even at the expense of some fidelity, can lead
to higher engagement and a stronger sense of presence, particularly in tasks requiring
high levels of interaction within the virtual environment.
13
2. Literature review
2.3.2 Immersion
The term "immersion" is used to describe the extent to which a virtual reality (VR) system
can engage the user’s senses, creating the perception of a real environment. Unlike the
presence, which is a psychological state, immersion is more about the objective properties
of the VR system itself [83]. It refers to the extent to which a VR system can envelop
the user’s senses, creating a convincing and engaging virtual environment. Immersion
is primarily concerned with the objective properties of the VR system, such as display
quality, field of view, and interactivity [63]. Achieving high levels of immersion involves
the use of stereoscopic 3D displays, which simulate depth perception, and head-mounted
displays (HMDs) that provide a wide field of view (FOV), enhancing the sense of being
surrounded by the virtual environment. These technical aspects are crucial as they allow
users to feel more “inside” the virtual world, thus increasing the realism and overall
experience [57].
In addition, interactivity plays a vital role in enhancing immersion. The ability for users
to interact naturally and intuitively with the virtual environment through advanced
motion tracking systems and adaptive VR controllers is essential. These systems track the
user’s movements and translate them into the virtual space in real-time, ensuring that the
virtual experience responds accurately to the user’s actions. This level of responsiveness
and interactivity makes the experience more immersive, as users can manipulate and
explore the virtual world as they would in the real world [57].
Recent studies have also highlighted the importance of multisensory feedback in enhancing
immersion. For instance, Shaw et al. [66] evaluated different forms of sensory feedback,
including auditory, physical resistance, and wind feedback, in an immersive exergame
environment. Their findings demonstrated that all three types of feedback increased the
level of immersion, with auditory feedback being particularly effective in enhancing the
user experience. The study also found that combining multiple forms of feedback, rather
than relying on a single type, produced the most immersive experiences. This research
underscores the significance of integrating diverse sensory inputs to create a more holistic
and engaging virtual environment.
2.3.3 Motion sickness
Motion sickness remains one of the major challenges in VR. It is caused by sensory
mismatches between visual inputs from VR headsets and vestibular systems that control
balance and spatial orientation [84]. This sensory mismatch can lead to symptoms such
as nausea, dizziness, and headaches. One crucial factor is optimizing frame rates, as low
or variable frame rates can exacerbate the discomfort experienced by users. Ensuring
a consistent and high frame rate helps to maintain a smooth and continuous visual
experience, which is critical in reducing the occurrence of motion sickness. Additionally,
the type of virtual environment and the level of user interaction can also influence the
severity of motion sickness. For instance, environments with rapid movements or poor
synchronization between head movements and visual feedback can significantly increase
14
2.4. Sensory feedback
the likelihood of discomfort [17]. Ensuring that the frame speed is stable and high,
reducing latency, and making the virtual environment more comfortable to navigate
are essential. In addition, movement algorithms that simulate natural movements, such
as teleportation instead of continuous movements, can help alleviate symptoms [4].
Providing users with control over their movements and the integration of rest frames
(static scenes) can also reduce the incidence of motion sickness.
2.3.4 Emotions
Emotions play a crucial role in VR experiences and have a significant impact on the user’s
engagement and overall effectiveness of the virtual environment. VR has a unique ability
to stimulate strong emotional reactions due to its immersive nature [58]. For example, VR
can create a sense of fear in horror games [49] or facilitate empathy in simulations that
seek to raise awareness of social issues. Emotions are stimulated in VR through various
mechanisms. Narrative content can create emotional connections by involving users in
compelling stories. Environmental design, including visual aesthetics and ambient sounds,
also contributes to the emotional tone of the experience. Recent research by Riches
et al. [62] supports this view, highlighting how VR can effectively be used for relaxation
and stress reduction among people with mental health conditions. Their systematic
review indicates that VR experiences designed to evoke specific emotional responses can
significantly enhance therapeutic outcomes by providing a controlled environment that
mimics real-life triggers. Understanding the emotional impact of VR is crucial to designing
experiences that are not only exciting but meaningful and memorable. According to
recent research, the sense of presence in a VR environment is closely tied to the emotional
responses users experience. This connection is particularly evident in scenarios designed
to provoke anxiety or fear, where the realism of the virtual environment can trigger
intense emotional reactions, similar to those experienced in real-life situations [25].
In summary, the main elements of VR – presence, immersion, motion sickness, and
emotions – are interdependent elements that collectively shape user experiences. For
the authors research these key factors will play a central role. From the initial design of
the application to the subsequent evaluation, these factors will prove to be of critical
importance.
2.4 Sensory feedback
Sensory feedback is a critical component of Virtual Reality that significantly enhances
the immersive experience by engaging multiple senses. It does not only improve realism
but also increases user engagement and satisfaction. This section explores the various
forms of sensory feedback in VR, including visual, auditory, tactile, and other sensory
modalities.
15
2. Literature review
2.4.1 Visual
Visual feedback is the cornerstone of VR, mainly dictated by the way users perceive the
virtual environment. Current visual feedback techniques include stereoscopic 3D and
high-resolution displays. Stereoscopic 3D creates depth perception by presenting slightly
different images to each eye, imitating the way human vision works [9]. This technology
is crucial to creating realistic and immersive VR experiences. High-resolution screens
with resolutions often over 4K provide clear and detailed images that increase the sense of
presence. Field of View (FOV) is another important factor; more extensive FOVs extend
peripheral vision and make the virtual world more inclusive and natural[40]. High-quality
visual feedback can make virtual environments more convincing and engaging, facilitating
deeper immersion. On the other hand, poor visual quality can destroy the feeling of
existence and cause discomfort and disinterest. Therefore, continuous advances in display
technology and rendering technologies are essential for the evolution of VR.
2.4.2 Auditory
Auditory feedback is a key element that significantly enhances immersion in virtual
reality. Advanced 3D audio technologies, such as binaural audio, play a crucial role in
creating a realistic and immersive sound experience. Binaural audio works by simulating
how sound waves interact with the human ear, creating a three-dimensional soundscape
that adjusts with the user’s head movements. This technique is particularly effective
in providing accurate spatial cues, thereby enhancing the sense of presence within the
virtual environment [10]. Broderick et al. [12] emphasize the importance of spatial audio
in modern games and virtual environments, highlighting how precise audio cues can
significantly improve user navigation and interaction within these spaces. The study
demonstrates that spatial audio not only enhances immersion but also provides critical
information that aids in orientation and environmental awareness, making the virtual
experience more engaging and realistic.
Binaural audio is a method of recording and reproducing sound that captures the way
humans naturally perceive audio in three dimensions. It simulates the natural listening
experience by using two microphones placed at the approximate positions of the human
ears. This technology takes into account the way sound waves interact with the human
head, ears and torso, creating a realistic spatial audio experience when reproduced
through headphones. The basic principle of binaural audio is the concept of interaural
time differences (ITD) and level differences (ILD). ITD refers to the slight difference in
the time it takes a sound to reach each ear, which helps the brain determine the direction
of the sound source. ILD is the difference in the sound pressure level reaching each ear,
which also helps to localize the sound. Together, these cues allow the brain to determine
the exact location of a sound source in the three-dimensional space around the listener [?
].
In addition, the Head-Related Transfer Function (HRTF) plays a critical role in binaural
audio. HRTF is a response that characterizes how an ear receives sound from a point
16
2.4. Sensory feedback
Figure 2.5: Explanation of binaural audio techniques [31]
in space, taking into account factors such as the shape of the ear (pinna), head and
body. When applied to a sound signal, HRTF filters it in a way that mimics how sound
would be altered by human anatomy, allowing precise simulation of sound coming from
any direction [61, 77]. The author thinks that binaural audio is essential for creating
immersive VR experiences as it greatly enhances the sense of presence and spatial
awareness in a virtual environment. It allows users to perceive sounds as if they were
happening in the real world around them, making the VR experience more realistic and
engaging. This technique is often used in conjunction with visual and haptic feedback to
create a fully immersive environment. More recent research by Kern and Ellermeier [45]
demonstrates that audio elements, such as soundscapes and movement-triggered step
sounds, play a crucial role in maintaining a high level of presence in VR environments.
Their study found that such auditory cues not only contribute to the spatial realism
but also enhance the immersive experience by aligning sound feedback with user actions.
Recent developments in binaural audio generation, particularly those using mono audio
sources, have further expanded its applicability. The process of converting mono audio
to binaural audio by utilizing depth and visual cues from the scene has shown promising
results in delivering a highly immersive auditory experience without the need for complex
recording setups. This method not only recreates the spatial properties of sound sources
but also enhances realism by incorporating depth information, which is critical for
accurate sound localization in VR environments [56]. Implementing such dynamic and
responsive auditory environments in VR can significantly elevate user participation by
providing contextual audio signals that enrich the overall sensory experience.
2.4.3 Tactile feedback
Tactile feedback, or haptic feedback, refers to providing physical sensations that cor-
respond to interactions in the virtual environment. State-of-the-art haptic technology
includes haptic gloves and vests that allow users to feel textures, shapes, and resistance,
thus enabling more natural and intuitive interaction with virtual objects. Devices like the
Manus VR and HaptX gloves use various mechanisms to simulate tactile sensations, such
as force feedback and vibrations. Haptic vests, such as Teslasuits, provide full-body feed-
back, allowing users to feel effects like temperature changes and other physical sensations.
17
2. Literature review
These vests use a series of actuators distributed throughout the body to provide precise
tactile feedback, enhancing the feeling of presence and immersion. Recent studies have
highlighted the importance of tactile feedback in VR, particularly in applications such
as surgical training and delicate operations. For example, research by Cheng et al. [18]
demonstrated that vibrotactile feedback could significantly improve task performance
in virtual environments, particularly in delicate tasks requiring precision and control.
The study found that adding vibrotactile feedback to visual and audio cues helped users
complete tasks more quickly, although it also led to increased pressure on manipulated
objects, indicating a potential trade-off between speed and delicate control. Further
advancements in haptic technology have explored the use of electro-tactile feedback as
an alternative to traditional mechanical feedback systems. Pamungkas and Ward [55]
developed an electro-tactile feedback system that delivers tactile sensations via mild
electric currents applied to the skin, offering a more compact and less cumbersome
solution compared to mechanical actuators. This system has been shown to enhance both
immersion and interactivity in VR environments by providing a diverse range of tactile
sensations without the bulkiness of traditional haptic devices. In medical training, the
role of haptic feedback has been extensively studied, particularly in laparoscopic surgery
simulators. A study by Våpenstad et al. [75] revealed that while haptic feedback is
considered crucial for simulating realistic tissue interactions, the implementation of such
feedback often introduces additional friction, which can detract from the realism of the
experience. Despite this, the emulation of haptic feedback remains a vital component in
training surgeons, as it helps in developing the necessary skills for real-world operations.
2.4.4 Other
While visual, auditory, and tactile feedback are the main sensory modes of VR, other
forms of sensory feedback, such as olfactory and gustatory, are attracting attention.
Olfactory feedback involves using smells to improve immersion. The study by Serrano
et al. [65] explored the use of olfactory stimuli, specifically lavender scent, in enhancing
virtual reality experiences. While the addition of smell did not significantly improve
relaxation or presence compared to visual and auditory stimuli alone, it showed potential
to enhance immersion. Participants exposed to the scent reported a trend towards greater
relaxation, suggesting that olfactory cues can enrich the sensory experience and emotional
engagement in VR when used alongside other stimuli. Generally, although less common,
gustatory feedback involves simulating taste sensations. While in the experimental phase,
researchers are exploring ways to incorporate taste into VR, potentially expanding its
applications in areas such as cooking training and virtual dining experiences [34].
In conclusion, sensory feedback in VR is crucial to creating immersive and stimulating
experiences. This thesis will focus on the evaluation of auditory and tactile feedback.
Since there are many studies available regarding the visual senses, while other sensory
feedback systems are not available and harder too evaluate. The clear focus lies on 3D
audio technology and tactile feedback through vibration.
18
2.5. Design principles
2.5 Design principles
Designing compelling and immersive virtual reality experiences requires an in-depth
understanding of design principles tailored to the unique characteristics of virtual environ-
ments. Key principles for immersive design emphasize realism, interactivity, consistency,
and feedback. Realistic rendering, including accurate object modeling, lighting, and
high-quality textures, is essential for creating believable environments that enhance
immersion. The concept of immersion is further strengthened when these elements are
combined with responsive and intuitive interactions, allowing users to feel truly part of
the virtual world. This can be seen in the “Claustrophobia Game,” which uses realistic
VR environments to treat anxiety disorders by simulating enclosed spaces like elevators
and MRI machines, providing a safe yet convincing experience for users [24]. Consistency
in visual and auditory styles throughout the VR experience is crucial for maintaining
immersion. Inconsistencies can disrupt the sense of presence and remind users of the
artificial nature of the environment. Immediate and appropriate feedback—both visual
and haptic—reinforces the sense of being in a responsive and interactive world, making
the virtual experience more immersive and engaging [92]. User-centered design (UCD) is
another critical aspect, placing the user’s needs at the center of the design process. This
involves creating comfortable and accessible experiences, which is particularly important
in VR to prevent discomfort such as motion sickness. For example, the design of VR
systems must account for ergonomics by optimizing the weight and balance of headsets
and ensuring that controllers are intuitive and comfortable to use. Additionally, provid-
ing adjustable settings for text sizes, control schemes, and options to minimize motion
sickness ensures that VR experiences are inclusive and accessible to a broader audience
[24]. Intuitive navigation is also a key factor in user-centered design, helping users move
through the virtual environment without confusion or frustration. This involves using
natural locomotion techniques like teleportation or room-scale movement, complemented
by clear visual cues for interaction points. In VR environments, effective navigation not
only enhances usability but also contributes to the overall enjoyment of the experience.
Designing virtual environments involves careful consideration of architectural principles
and environmental storytelling. Spatial layout should facilitate easy navigation, with
logical layouts and clear landmarks helping users orient themselves naturally. Proper
scaling of objects and environments is vital to maintaining realism; disproportionate
elements can break immersion and affect the user’s perception of space. Environmental
factors like lighting, sound, and atmospheric effects play a significant role in setting the
mood and enhancing the narrative, creating environments that evoke specific emotions
and enhance the overall experience. Wagener et al. [76] emphasize the importance of
designing VR environments that support emotion regulation by integrating intuitive
navigation with thoughtfully designed spaces. Their research suggests that by considering
emotional impact during the design process, VR applications can better facilitate users’
emotional well-being, making the environment both engaging and therapeutic.
Case studies of effective VR design provide valuable insights into successful approaches.
For instance, “Job Simulator” by Owlchemy Labs excels in interactivity and humor,
19
2. Literature review
creating an engaging and intuitive environment with a consistent, exaggerated, cartoonish
design that enhances the immersive experience [74]. “Google Earth VR” is another great
example, where the user gets immersed in a another world, in this case a representation
of the real world. The intuitive navigation makes it easy to navigate through a 3D version
of the earth, which allows the user to immerse themselves in any specific location. While
the interaction is limited to navigating, is is interesting enough to keep users immersed
for a long time period. [85]. “The Climb” by Crytek leverages realistic graphics, precise
motion tracking, and immersive sound design to create a thrilling experience, with careful
consideration of physical interaction and feedback enhancing the sense of presence and
adventure [35].
In summary, effective VR design requires a balance of realism, interactivity, and user-
centered considerations. By focusing on ergonomic comfort, accessibility, and thoughtful
environmental design, creators can develop VR experiences that are immersive, engaging,
and inclusive. These principles, coupled with lessons from successful applications, provide
a roadmap for creating compelling VR environments that resonate with users.
2.6 Related work
Numerous projects and studies have explored the critical role of sensory feedback in
enhancing immersion and presence in virtual reality (VR). This section reviews key
contributions to the field, focusing on how they address multisensory feedback and the
challenges of achieving high levels of user engagement through tactile and auditory
stimuli.
The work of Slater and Wilbur [68] laid the groundwork for understanding the concept
of presence in VR environments, emphasizing the importance of consistent and realistic
sensory input to foster a deep sense of ’being there’ in virtual spaces. Their study showed
that even subtle inconsistencies between sensory modalities, such as visual and auditory
stimuli, can disrupt presence, highlighting the need for seamless integration of sensory
feedback. This principle provides an important foundation for their research, particularly
in the context of ensuring that both binaural audio and haptic feedback are tightly
synchronized to create a cohesive virtual experience. More recently, McMahan et al. [52]
investigated the impact of high-fidelity visual and auditory feedback on user immersion,
showing that higher quality sensory input significantly enhances the sense of presence.
While their study focused primarily on visual and auditory cues, it did not explore the
potential synergy between these and other sensory modalities, such as haptics. This
highlights a gap in the literature where multimodal integration, particularly between
binaural audio and haptic feedback, has not been fully explored. Her research aims
to address this gap by investigating how the combination of these two sensory inputs
can enhance user immersion and engagement in VR environments. The 2015 Oculus
Spatial Audio Research Study further underscores the importance of accurate 3D sound
in VR. The study found that spatial audio significantly improves users’ ability to orient
and navigate in virtual environments by providing critical spatial cues [64]. However,
20
2.6. Related work
the study focused primarily on auditory feedback in isolation, without considering how
tactile feedback could complement the auditory experience to create a more immersive
environment. This gap suggests that further research is needed to evaluate the combined
effect of binaural audio and haptic feedback, which your thesis will investigate in the
context of user engagement and presence. The Virtuix Omni project explored the role of
whole-body motion tracking in enhancing physical participation and presence in VR. Their
findings suggest that allowing users to walk and move naturally in virtual environments
enhances immersion. While this study focuses on kinesthetic feedback, it highlights
the broader importance of integrating multiple sensory modalities to enhance the VR
experience. The mentioned research will extend this concept by specifically evaluating
the interplay between binaural audio and haptic feedback, with the aim of understanding
how the integration of auditory and haptic cues influences user immersion without the
need for full-body movement systems [72].
Similarly, the “Claustrophobia Game” project investigated the effects of environmental
sensory stimulation on users with anxiety disorders using accurate tactile and auditory
feedback to create a calming, controlled environment. The study demonstrated that
controlled exposure to simulated confined spaces, enhanced by accurate tactile and
auditory feedback, could help reduce anxiety symptoms, showcasing the therapeutic
potential of VR environments when designed with sensory stimulation in mind [60].
Another project, “The Climb” by Crytek, focused on the use of realistic graphical and
auditory stimuli combined with precise motion tracking to create an intense, physically
engaging experience. The study found that users experienced a heightened sense of
presence and physical engagement, indicating that multisensory feedback is key to
creating compelling and immersive VR experiences [2].
The Virtual Jumpcube, developed by Eidenberger in 2015, offers a case study of how
integrating multiple sensory modalities can enhance immersion and user experience in
virtual reality environments. This system combines audiovisual feedback with olfactory
and haptic stimuli, allowing users to engage in a fully immersive jumping and flying
experience. Unlike traditional VR systems, the Virtual Jumpcube employs complex
mechanical setups to simulate physical sensations, such as wind, force, and suspension,
which are carefully synchronized with the virtual environment. Eidenberger’s primary
goal was to investigate how the addition of olfactory and tactile stimuli can heighten
immersion, excitement, and minimize motion sickness during VR experiences. The results
of his experiments demonstrated that participants reported significantly higher levels of
immersion when haptic and olfactory feedback were included, especially when these stimuli
were carefully aligned with visual and auditory cues. For example, sensations such as wind
or water spray, delivered during virtual skydiving, were found to increase engagement by
providing realistic tactile feedback. Participants overwhelmingly responded positively to
these stimuli, citing them as crucial to the overall experience [27].
The study by Bruun-Pedersen et al. [15] investigated the influence of multimodal feedback,
particularly auditory and haptic stimuli, on perceived exertion in a virtual reality
exercise context. The findings offer valuable insights for the design of immersive virtual
21
2. Literature review
environments. The study involved participants utilizing a stationary exercise bike within
a simplified desert virtual reality environment, with variations in auditory and haptic
feedback to simulate different exertion levels. Contrary to the hypothesis that such
feedback would significantly alter perceived exertion, the results indicated no substantial
differences across the conditions tested. However, the qualitative responses indicated
that some participants perceived alterations in mechanical resistance and found the
experience inconsistent with their real-world expectations, primarily due to the absence of
gravity simulation in the virtual environment. This outcome emphasizes the necessity of
aligning virtual sensory feedback with real-world physics to enhance user immersion and
efficacy in VR applications, particularly in the context of exercise and rehabilitation. The
study highlights the importance of further research on multimodal feedback to elucidate
its role in shaping user experience in VR, in line with ongoing efforts to investigate
under-researched areas of sensory feedback and their potential to influence engagement
and immersion in virtual environments [15].
Figure 2.6: The Setup used by Bruun-Pedersen [14]
These studies and projects collectively highlight the critical role of multisensory feedback
22
2.6. Related work
in achieving immersive VR experiences, demonstrating the importance of integrating
visual, auditory, and tactile stimuli to create more convincing and engaging virtual
environments. However, the combination of binaural audio and tactile feedback in
particular has hardly been researched, which contributes to the ambition to investigate it
further in this thesis. The research by Bruun-Pedersen et al. provides a good starting
point, highlighting both the possibilities and limitations of combining audio and tactile
feedback. Their study will serve as a framework for comparing results, as it uses a similar
system ( 2.6 ) to this study. In the following chapter the design of the study as well as
the requirements and the design of the VR experience will be presented.
23

CHAPTER 3
Design and concept
The following describes the conception and design of the VR system. The system is
derived from the project idea "Phantastic Sound Experience" by Professor Dr. Horst
Eidenberger [1]. The requirements that form the basis of the design are described first,
followed by the research and concept that led to the implementation. To effectively
evaluate the impact of auditory and tactile feedback on immersion and presence in virtual
reality, the system design and implementation must meet several key requirements. These
requirements ensure that the VR environment not only supports the necessary sensory
modalities but also provides a stable and reliable platform for comprehensive testing and
evaluation.
3.1 Requirements
3.1.1 Uniformity of participant experience
The VR experience should be designed to ensure that each participant experiences an
equivalent sequence of events, interactions, and sensory stimuli. This uniformity is
critical for generating comparable evaluation data across all participants and directly
supports the research objective by ensuring that any variation in participant responses
can be attributed to the sensory feedback being tested, rather than discrepancies in
their individual VR experiences. As Bowman et al. [11] argue, consistency in the VR
environment helps to control the variables that might otherwise impact user experience
and interaction. Similarly, Slater and Wilbur [68] emphasize that a controlled and
consistent virtual environment is essential for studying presence and immersion, as
it allows researchers to isolate the effects of specific sensory inputs. By maintaining
consistency in the virtual experience, the variability that could arise from different
participant interactions is minimized. This consistency is particularly important when
evaluating the effects of auditory and tactile feedback on immersion and presence, as
25
3. Design and concept
it ensures that differences in participants’ responses can be attributed to the sensory
modalities being tested, rather than to variations in the VR experience itself. Thus,
the principle of uniformity directly supports the reliability and validity of experimental
results, allowing researchers to draw more accurate conclusions about the impact of
specific sensory inputs on user experience.
3.1.2 Balanced and immersive virtual environment
The virtual environment should strike a careful balance between being engaging enough
to promote immersion and neutral enough not to overshadow other sensory inputs. This
balanced design supports the research objective of investigating how auditory and tactile
feedback influence immersion by ensuring that sensory inputs are the primary focus.
The design should be compelling enough to maintain a high level of presence but subtle
enough to allow auditory and haptic feedback to be the primary focus of evaluation.
Potter et al. [59] emphasize that achieving this balance is crucial because both visual and
audio rendering play significant roles in shaping user immersion. Their study shows that
while high-resolution visuals enhance presence, spatial audio—especially when augmented
with room acoustics—can have a similarly profound impact. This finding highlights
the need for a nuanced approach to designing VR environments that do not dominate
but rather complement sensory stimuli being tested. This balance is essential to ensure
that the sensory stimuli being tested are the dominant factors influencing participant
immersion and presence, thereby supporting the overall research objectives. In addition,
the environment should include adjustable elements such as lighting and sound levels that
can be tailored to meet the specific needs of different experimental conditions, further
aligning with the modular design of the system.
3.1.3 Integration of sensory feedback
The VR system should seamlessly integrate hardware and software components to support
visual, auditory, and tactile feedback to ensure a cohesive and immersive experience.
Effective VR design, as suggested by Jerald [43], requires the tight integration of these
elements to enhance user immersion and presence. This includes the development of a
customizable virtual environment capable of controlling and synchronizing sensory output.
The hardware configuration must include spatial audio systems and haptic devices, both
of which play a critical role in simulating real-world sensory experiences. Synchronization
of these components is essential to deliver auditory, tactile and visual stimuli in harmony
to create a seamless sensory environment. According to Cummings and Bailenson [22]
the synchronization of sensory modalities is fundamental to improving the overall sense of
presence and ensuring that the virtual experience feels natural and convincing. Sensory
feedback needs to be perceived as an integral part of the virtual environment, rather
than a separate layer, so the interplay between these stimuli is key to achieving high
levels of immersion. Han et al. [33] further emphasis the importance of multisensory
feedback integration, highlighting systems such as ’Haptic Around’ that combine tactile
feedback mechanisms to simulate environmental factors such as wind, heat and humidity.
26
3.1. Requirements
Their findings show that a range of tactile sensations enhances realism by making sensory
feedback feel connected and authentic. Minimizing latency or mismatch between these
inputs is critical to maintaining the illusion of reality and ensuring the immersive quality
of the VR experience.
In addition, the system should allow the intensity and timing of sensory feedback to
be adjusted to suit specific experimental conditions or participant preferences. This
flexibility enhances the adaptability of the system and supports broader research goals
by enabling tailored sensory experiences in different experimental setups.
3.1.4 Participant centered design and accessibility
Given that many users may be new to virtual reality, the system must be designed
with a strong emphasis on participant comfort and accessibility. This requirement is
closely related to the balanced virtual environment, as an overwhelming or overly complex
environment could detract from the user experience. The system should minimize potential
overwhelm by providing an intuitive interface and gradually introducing participants
to the VR environment. Accessibility features, such as customizable haptic feedback
and adjustable control schemes, must be included to accommodate users with varying
physical abilities. Additionally, the ergonomic design of the system is essential to prevent
physical fatigue and allow participants to remain in the virtual environment for extended
periods of time without discomfort. Yao and Kim [91] highlight that maintaining a
high level of presence and engagement in VR environments is significantly influenced
by physical comfort and ease of interaction. Their research shows that a well-designed,
user-friendly VR setup not only enhances immersion but also encourages physical activity,
which can further elevate the sense of presence. By ensuring that the system is user-
friendly and accessible, the likelihood of distraction due to physical or cognitive strain
is minimized, which in turn supports the overall goal of maintaining a consistent and
immersive experience for all participants.
3.1.5 Optimised test duration and engagement
User testing sessions should be designed to optimize participant engagement and data
collection. This requirement is related to participant-centered design, as the duration of
the test has a direct impact on the user’s comfort and willingness to engage with the VR
system. The duration of the test should be long enough to allow participants to fully
engage with the VR experience and provide meaningful feedback, yet short enough to avoid
boredom or fatigue. The experience should also be designed to progressively introduce
participants to the VR environment and sensory feedback mechanisms, allowing them
to adapt before more complex interactions are required. This progressive engagement
ensures that participants are neither overwhelmed nor under-stimulated, maintaining
a high level of immersion throughout the session. Incorporating an adaptive testing
mechanism that adjusts session length based on participant engagement levels can further
27
3. Design and concept
enhance the quality of data collected and ensure that the experience remains both
engaging and informative [16].
3.1.6 Evaluation tools
To assess the effectiveness of the sensory feedback, the evaluation with participants
should be possible directly after testing and include both quantitative and qualitative
measures. Quantitative assessments should employ standardized scales like the Igroup
Presence Questionnaire and the Immersion Tendencies Questionnaire to objectively
measure presence and immersion. Additionally, the system should facilitate the collection
of qualitative feedback through open-ended questions that allow users to describe their
experiences and suggest improvements. Kim et al. [46] demonstrated the importance
of using multimodal sensory feedback, such as auditory, visual, and haptic cues, in
enhancing user performance and presence in VR environments. Their study found
that the effectiveness of these sensory modalities could be better understood through
comprehensive evaluation methods that combine both quantitative metrics and qualitative
insights. These tools are essential for gathering meaningful data that will inform the
development of more immersive VR systems by highlighting how different sensory inputs
contribute to the overall user experience.
3.2 Design approach
Having identified the key requirements for enhanced immersion in virtual reality through
multisensory integration, this section outlines the chosen design approach for the system
and builds a bridge to the employed methodology.
3.2.1 Experience design
The design approach focuses on creating a virtual world experience that prioritizes
objectivity instead of interactivity, as outlined in requirement . Therefore it was decided
to design a passive virtual environment, instead of developing a game, which often
involves complex interactivity and tasks that can vary significantly between users. In
this environment, participants take on the role of observers or ’passive users’ rather
than active players. This choice ensures that all participants share a similar experience,
allowing them to focus on the sensory aspects of the environment without being distracted
by game play mechanics or objectives.
3.2.2 Physical device integration
The challenge of connecting the virtual and real worlds through tactile feedback was
a major focus of this design approach. While many VR controllers on the market
provide some form of tactile feedback, the intensity and realism of these sensations
are often limited. There are several commercially available solutions to enhance tactile
feedback, such as haptic gloves and suits, which provide a more immersive experience by
28
3.2. Design approach
simulating the sense of touch. Examples include the TeslaSuit, which provides full-body
haptic feedback, and the HaptX gloves, which provide detailed tactile sensations through
microfluidic technology. However, these solutions are typically expensive and may not be
accessible to all users or practical for large-scale implementation in research or consumer
settings. Given these limitations, it was decided to design and build a custom physical
device specifically tailored to the needs of this virtual experience. The idea was to
create a physical component that could be directly represented in the virtual world,
allowing real-world vibrations and tactile sensations to be seamlessly translated into
the VR environment. This approach not only reduces costs, but also allows for a more
controlled and targeted application of haptic feedback, enhancing the overall immersive
experience. This also enables the possibility of incorporating accessibility in the design,
supporting the requirements of 3.1.4. The design process involved brainstorming and
sketching different concepts for the physical device. The aim was to develop a solution
that could effectively simulate the sensation of touch in a way that was both realistic and
synchronized with the virtual environment. Several design iterations were considered,
each exploring different methods of integrating the physical and virtual components. The
chosen design focuses on a device that can provide adjustable levels of vibration and
pressure, allowing for a range of tactile experiences that can be fine-tuned to suit specific
scenarios within the virtual world. The device is intended to be simple yet effective,
using readily available materials and components to keep costs down while maximizing
the impact on the user experience. The design also takes into account the ergonomics
and comfort of the user, ensuring that the device can be used for long periods of time
without causing discomfort or fatigue.
Figure 3.1: Sketch, first draft of the physical construction
The implementation details of the device, including its construction and integration into
the virtual environment, are described in more detail in the following section 4.3.1. This
section will cover the technical aspects of the device, including the specific materials and
components used, as well as the methods for synchronizing the physical and virtual worlds
to achieve the desired level of immersion. Through this design approach, the system
aims to provide a more immersive and objective virtual reality experience, enhancing the
29
3. Design and concept
user’s sense of presence and engagement in the virtual world.
3.2.3 Multisensory feedback synchronization
The main idea of the sensory feedback was to link events in the virtual world with
the corresponding senses. This ensures a synchronized line of events for the sensory
feedback. To ensure a seamless experience for the player in virtual reality, it is essential
that the audio and vibration correspond to the same location. In order to achieve this
synchronization, the vibrations should be linked to the audio. The sequence of sensory
feedback is be represented in the following flowchart:
Figure 3.2: Illustration of synchronized feedback loop
So the event in the virtual world determines the volume of the audio, and the volume of
the audio determines the strength of the oscillation. This system ensures that all sensory
modalities are correctly aligned, providing a smooth and immersive user experience.
Synchronizing these inputs can present technical challenges, particularly in minimizing
latency between audio and tactile feedback. Drawing from research on the importance
of synchronized sensory feedback as outlined in 3.1.3, the system leverages advanced
processing techniques to minimize delay. For example, when an object hits the virtual glass
cube, the corresponding sound and tactile vibrations are delivered almost instantaneously.
How this can be achieved will be shown later in the section 4.3.3 of the implementation
chapter.
3.2.4 Test design
The test structure was designed to be consistent and repeatable, ensuring a uniform
experience for each participant. Users first received an introduction to the environment to
help them acclimate to the virtual world. During the test they observe objects interacting
with their environment from within the glass cube, experiencing synchronized audio and
tactile feedback. The test lasts five minutes each, with a sequence of events designed
to gradually increase the complexity and intensity of the stimuli. The controlled, non-
interactive environment ensures that each participant has a similar experience, supporting
30
3.3. Conception of the VR experience
the requirement for uniformity. This is necessary to provide a clear and consistent test
procedure, ensuring robust data collection across multiple participants. The test design
supports the aim of the study to isolate the effects of multisensory feedback on immersion.
3.2.5 Evaluation plan
The evaluation of the sensory feedback system includes both qualitative and quantitative
methods to ensure a thorough analysis of its effectiveness. Key metrics for evaluation
include user reported immersion and presence inspired by standardized questionnaires
such as the Igroup Presence Questionnaire and the Immersion Tendencies Questionnaire.
Qualitative feedback was collected through open-ended questions, allowing participants
to describe their experience and highlight any potential discrepancies between sensory
modalities. A statistical analysis was be employed to identify the impact of sensory
feedback on the level of immersion experienced by participants, thereby ensuring the
accurate analysis of the data. This comprehensive evaluation plan provides a clear under-
standing of the system’s ability to enhance presence and immersion through multisensory
integration.
3.3 Conception of the VR experience
The objective of this "Phantastic Sound Experience" is to investigate the potential of
combining auditory and tactile feedback to significantly enhance user immersion and
presence in a virtual reality environment. This section presents the concept that was
implemented and the design of the system in order to include sensory feedback. Moreover
it displays how the design choices support the research objectives. The following will
detail how the VR experience is designed to serve as a basis for the subsequent evaluation
and analysis.
3.3.1 Game concept
In the virtual environment, the user is placed in a glass cube which is placed in the
center of the room. This room, designed to respond dynamically to the accompanying
music, gradually fills with a variety of objects. These objects move through the air and
collide with the glass cube, producing sounds that trigger the vibration, synchronizing
the physical vibrations with the virtual events. What differentiates this system from
aforementioned projects is the decision to use a non-interactive, passive environment.
Unlike traditional VR systems that rely on user actions and interactivity, this design
focuses on providing a consistent sensory experience where all users observe and experience
the same sequence of events. This not only ensures uniformity, but it also allows the
study of sensory feedback in a controlled setting, free from the variability that interaction
might introduce. The synchronization between auditory and tactile feedback is crucial to
maintaining immersion, as it ensures that the sensory input matches the visual stimuli,
creating a cohesive and believable experience. This approach directly supports the
31
3. Design and concept
requirement for 3.1.3, where the interplay between sensory modalities must be seamless
to avoid disrupting the user’s sense of presence.
This non-interactive design marks a departure from research such as that of Bruun-
Pedersen et al, where VR systems have been used to modulate physical effort through
user interaction in a more active context. By concentrating on a passive, observational
setting, the system identifies the impact of sensory feedback (sound, touch and sight)
on user immersion, eliminating the potential for confounding variables introduced by
user interaction. This approach provides a more controlled and comparable dataset
across participants, emphasizing the sensory experience itself as the primary factor
influencing immersion. In addition, the environment incorporates elements that respond
to the audio cues, with the volume of the sound depending on the proximity of the user
and the surrounding components. This dynamic interaction between the user and the
environment is designed to maintain engagement and immersion without overwhelming
the user, in line with the requirement for a balanced virtual environment as described
in 3.1.2. By calibrating the complexity of the environment, the design ensures that
it remains engaging without overshadowing the primary focus on auditory and tactile
feedback. The interactive experience is characterized by an increasing level of animation
within the space, with objects interacting more intensely with the glass cube as the
musical accompaniment progresses. This results in a moment when the cube, with the
user inside it, begins to float in space. This escalation is intended to create a compelling
and dynamic experience that fully engages the user’s senses, potentially creating a lasting
sense of presence and immersion. This design strategy supports the requirement for
optimized test duration and engagement from 3.1.5, ensuring that participants remain
engaged throughout the experience, allowing for a thorough and meaningful evaluation
of their responses to the sensory inputs. In addition, the dynamic behaviour of objects
and sound cues further differentiates this system from existing research. Rather than
relying solely on user interaction to drive immersion, our design uses the movement of
objects and their corresponding sounds to maintain engagement. For example, when
objects collide with the glass cube, the sounds produced are not only spatially accurate,
but also trigger corresponding tactile feedback in real time, seamlessly and dynamically
aligning auditory, visual and physical sensations. This synchronization ensures a high
level of immersion, even in the absence of direct user interaction, and offers a different
perspective on how virtual environments can maintain user engagement through sensory
stimuli alone.
The decision to maintain a neutral yet engaging environment is also critical. The virtual
world is kept simple yet realistic, with enough detail to create a strong sense of presence
without overwhelming the user. This approach helps to ensure that the environment
does not distract participants from the primary focus of sound and tactile feedback,
and that their attention remains focused on the sensory elements being studied. This
simplicity supports the uniformity mentioned in 3.1.1 as it minimizes the risk of external
variables affecting the results, thus ensuring consistency across different users. Interactive
elements are incorporated not only to maintain participant interest, but also to provide a
32
3.3. Conception of the VR experience
comprehensive assessment of presence and immersion. By engaging users through these
interactions, the design allows for a more robust assessment of how auditory and tactile
feedback contribute to the overall VR experience. This is in line with the principle of
user centered design as mentioned in 3.1.4, which emphasizes the importance of creating
a system that is both accessible and engaging, allowing users to fully immerse themselves
in the virtual environment without physical or cognitive strain.
Overall, this design approach ensures that the virtual world remains dynamic and
engaging, while emphasizing the importance of audio and tactile feedback in creating
a realistic and immersive VR experience. The combination of a neutral yet interesting
environment, strategic environmental elements and interactive dynamics supports a
comprehensive assessment of presence and immersion in virtual reality. By integrating
these design choices with the core requirements, the system is able to provide valuable
insights into the role of multisensory feedback in enhancing VR experiences.
3.3.2 Design of the environment
Figure 3.3: Front view Bofill’s of La Muralla Roja [19]
The environment is designed as a floating virtual architectural structure in the sky
inspired by Ricardo Bofill’s works, especially La Muralla Roja (3.3). This structure
offers a neutral but interesting environment that aligns with the principles of simplicity
and realistic design as defined in 3.1.2. One of the first design sketches in blender
is shown in 3.4. Within this architectural space, elements are strategically placed to
ensure that the room does not appear empty and improve the spatial experience. At
the center of this virtual environment is a concrete-based glass cube where participants
sit. This central position helps to focus interaction and sensory experiences around the
participant, strengthening their sense of presence in the virtual world. To create dynamic
and engaging experiences, the virtual world includes various interactive elements. The
room is gradually filled with 3D spheres that collide with the glass cube. These collisions
produce sound and vibration, which are key to assessing participants’ immersion and
presence. This system allows constant interaction without requiring the participant to
move around in the real world, thus maintaining a balance between engagement and
33
3. Design and concept
practicality. As the number of spheres increases and interacts more frequently with the
cube, they create a continuous and evolving sensory experience. Furthermore, sphere
movement and related audio sources improve spatial audio awareness and contribute
further to environmental immersive quality.
Figure 3.4: Explorative rendering of virtual world
3.3.3 Audio
As described in 2.4.2, audio plays a crucial role in enhancing realism and immersion
to create an interactive virtual world. In reality, there is always a certain amount of
background noise, and reproducing this in the virtual environment helps to create an
atmosphere that feels alive and engaged. The soundscapes are designed to evoke the
feeling of being in the sky and contribute to the overall feeling of being in another
dimension or virtual world. This involves creating a soundscape that evokes the vastness
and serenity of the sky. In addition, various sound effects need to be synchronized with
interactions in the virtual world. For example, each collision between spheres and objects
generates a sound based on the physical properties of the objects, ensuring that the audio
feedback matches the visual and haptic experience. The placement of sound sources
in the virtual world is critical to achieving realistic and immersive sound. Each sound
must be placed and calibrated to match the visual elements and physical interactions in
the environment. This realistic sound placement enhances the user’s sense of space and
presence. Multiple tests and adjustments ensure that the soundscape is harmoniously
integrated into the virtual world. For example, as spheres move and collide in space, their
associated sounds accurately reflect their position and movement, providing a coherent
and immersive auditory experience. Implementing 3D audio in the VR environment
requires advanced audio technologies to create a realistic soundscape. A spatializer is used
to generate three-dimensional sound, allowing users to perceive the direction, distance
and movement of sounds in virtual space. Binaural audio technology simulates how
the human ear naturally hears sounds from different directions, enhancing immersion.
Dedicated audio engines, such as Unity’s, enable real-time processing and synchronization
34
3.3. Conception of the VR experience
of audio signals with visual and tactile feedback. These tools ensure precise control of
spatial audio effects, creating an immersive and realistic sound environment [41].
Comparing the authors approach to auditory feedback with Bruun-Pedersen et al.’s
research, several similarities emerge. Both systems aim to synchronize auditory feedback
with physical events in the virtual world. In their virtual exercise study, Bruun-Pedersen et
al. used dynamic auditory feedback to reflect user effort, whereas in the "phantastic sound
experience" sound is synchronized with environmental events such as object collisions.
However, while the work focused on modulating auditory feedback to influence perceived
physical exertion, the system uses sound to enhance the user’s sense of immersion,
allowing participants to focus on sensory stimuli and environmental interactions rather
than physical performance [15]. This approach aligns with our research objective to
explore how sensory feedback contributes to immersion by creating an environment where
auditory feedback complements visual and tactile inputs to support a seamless sensory
experience.
The combination of audio and tactile feedback is critical to creating a cohesive sensory
experience. Just as spatial audio provides directional and distance cues, tactile feedback
must be seamlessly integrated to reflect the physical sensations associated with these
sounds. This link ensures that when users hear objects interacting in virtual space, such
as collisions, they feel corresponding vibrations, adding a layer of physical realism to the
auditory experience. The following section explores how tactile feedback, in combination
with audio, plays a central role in creating a believable virtual world.
3.3.4 Tactile feedback
Tactile or haptic feedback enhances immersion in virtual reality by providing physical
sensations that correspond to the user’s interactions in the virtual world. This feedback
bridges the gap between the virtual and physical experience, making virtual environments
more believable and engaging. Several projects have integrated tactile feedback into VR,
such as tactile gloves and vests that simulate touch and force. The HTC Vive controllers,
for example, use vibration motors to give users tactile responses during interaction. These
design principles ensure that tactile feedback matches the virtual interaction, enhancing
realism and user engagement [67, 23].
For the "Phantastic Sound Experience", tactile feedback is implemented using vibration
motors to simulate physical sensations. The buttkicker module, a key component,
produces vibrations that simulate various interactions, such as the collision of virtual
spheres with the glass cube. This hardware is critical to providing realistic tactile
feedback that enhances the overall immersive experience. The synchronization of tactile
feedback with virtual events is essential for immersion. For example, when virtual spheres
collide with the glass cube, vibration motors are activated to provide tactile feedback
corresponding to the impact. This synchronization ensures that physical sensations
match virtual actions, enhancing realism and immersion. Some reference projects,
such as Adelsberger’s, have demonstrated the benefits of integrating tactile feedback
35
3. Design and concept
to significantly improve user participation and presence. In the " Phantastic Sound
Experience", virtual interactions are closely linked to vibration feedback, ensuring a
consistent and immersive experience for participants [3]. Bruun-Pedersen et al.’s study
also integrated tactile feedback, using vibration motors to simulate physical effort during
a VR cycling exercise. While both systems use tactile feedback to enhance immersion,
Bruun-Pedersen focused on using vibration to alter perceived physical exertion. In
contrast, our project uses tactile feedback to mirror virtual interactions, such as object
collisions, to enhance the user’s sensory immersion. Both approaches highlight the
importance of synchronizing tactile stimuli with other sensory inputs, although the
end goals are different: where Bruun-Pedersen targets perceived exertion, this project
emphasizes sensory coherence to support the user’s sense of presence and immersion in
the virtual environment [15].
As outlined in the previous section on audio, tactile feedback works in together with sound
to create a fully immersive experience. Each tactile cue is matched with its auditory
counterpart, ensuring that users not only hear but also feel the interactions occurring
around them, reinforcing the sense of presence in the virtual world.
3.3.5 System integration and synchronization
As mentioned before in 3.2.3 and the two previous sections, the importance as well
as the challenge arises in integrating the sensory feedback together. The system must
provide synchronized multisensory feedback, meaning that audio and tactile stimuli are
aligned with visual cues in the virtual environment. This synchronization between sensory
modalities is central to the studies approach, as it allows us to study immersion in a
non-interactive context, focusing on how well sensory cues alone can engage users. The
design prioritizes the interplay between these sensory modalities, ensuring that they
complement rather than compete with each other. For example, when a user observes
an event in the virtual world that would naturally produce both sound and a physical
sensation - such as an object hitting the glass cube - the system ensures that auditory
and tactile feedback occur simultaneously. To achieve this, the system uses advanced
audio processing techniques to spatialize sound accurately within the virtual environment,
creating realistic soundscapes that enhance the user’s sense of space. At the same time,
tactile feedback delivered through the custom-built physical device is synchronized with
these sounds to provide physical sensations that match the intensity and character of the
auditory cues. In both cases, synchronization between sensory modalities is crucial, but
the research objectives are different. While other mentioned studies explore the role of
feedback in managing physical strain, this project focuses on creating a sensory experience
that supports research into the impact of multisensory feedback on user immersion and
presence. With the concept of the "Phantastic Sound Experience" clearly defined, the next
step was to translate these design choices into a functional system. The core objective of
combining auditory and tactile feedback to enhance user immersion has been established
through integration of sensory modalities, a neutral yet engaging environment, and
synchronized feedback mechanisms. The conceptual framework provides a sound basis for
36
3.4. Scientific process
evaluating how these sensory inputs contribute to immersion and presence, but the success
of this study depends on its technical realization. The following chapter focuses on the
implementation phase, detailing how the hardware and software components of the system
come together to create the immersive experience envisioned in the design. This includes
the development of custom tactile devices, the synchronization of audio-visual-tactile
feedback and the use of advanced audio processing technologies.
3.4 Scientific process
3.4.1 Engineering approach for VR
The engineering process serves as the foundation for the system’s development, providing a
structured framework for integrating the disparate components that collectively constitute
the virtual reality experience. From a theoretical perspective, the engineering approach
emphasizes the construction of a robust system that seamlessly integrates visual, auditory
and tactile feedback, thereby enhancing user immersion. The system’s design incorporates
well-established virtual reality development platforms, including the Unity engine and
Steam Audio. These have been selected on the basis of their capacity to deliver high-
quality visual and spatial audio feedback. The flexible architecture of Unity permits the
real-time rendering of dynamic three-dimensional environments, while Steam Audio offers
sophisticated spatialization techniques that permit users to hear sounds from different
directions, thereby enhancing the realism of the virtual world. A significant element of
the engineering methodology is the incorporation of haptic feedback, utilizing haptic
technologies to impart a sense of physical engagement within the virtual environment. The
plan for the tactile component entails the utilization of vibration motors that translate
low-frequency audio signals into physical sensations, thereby reinforcing the user’s sense
of presence. It is anticipated that this multisensory integration will enhance the system’s
capacity to fully immerse participants by synchronizing haptic responses with audio and
visual cues. From a theoretical perspective, this engineering approach aims to create
a cohesive virtual environment where sensory feedback systems work in harmony. It is
essential that calibration of the various elements is carried out correctly, particularly in
ensuring low latency and synchronization between audio and tactile stimuli, in order to
avoid breaking the immersive experience.
3.4.2 Prototyping process
The system will adopt an evolutionary prototyping approach, thereby enabling the
design to evolve in an iterative manner through cycles of testing, user feedback, and
refinement. Evolutionary prototyping is a widely accepted method in software and system
design, particularly in complex, interactive environments like VR, where user feedback
is critical for refining the user experience. This methodology enables the system to
commence with a fundamental prototype that incorporates the essential components,
including visual rendering, spatial audio, and fundamental haptic feedback. This is
followed by the implementation of incremental enhancements. The prototype will initially
37
3. Design and concept
be evaluated in a controlled setting to ascertain the functionality of the fundamental
systems, including the physics-based interactions, sound propagation, and tactile feedback
mechanisms. Through evolutionary prototyping, the system is expected to evolve into
a more immersive and responsive VR environment. Each iteration will aim to address
specific design challenges identified during testing, ensuring that the final system provides
a polished and seamless multisensory experience. The advantage of this method is its
flexibility - it allows for constant adjustments without requiring a fully complete system
at the outset, allowing for continuous improvement as the system evolves. Through
evolutionary prototyping, the system is expected to evolve into a more immersive and
responsive VR environment [86].
3.4.3 Evaluation methods
The evaluation of the "Phantastic Sound Experience" will be guided by qualitative
evaluation methods, with a particular focus on understanding users’ subjective experiences
within the virtual environment. Theoretical models from human-computer interaction
and virtual reality research, such as Witmer and Singer’s work on presence and immersion,
will inform the evaluation process. In the planned evaluation, participants will complete
structured questionnaires based on established scales such as the Presence Questionnaire
and the Immersive Tendencies Questionnaire. These instruments are designed to capture
users’ subjective responses to various aspects of the VR experience, including their sense
of presence in the virtual world and their level of sensory engagement. For example,
questions will assess how fully the environment engages the user’s senses or how aware
they remain of their physical surroundings during the VR experience. A Likert scale
will be used for most questions, allowing users to rate their experience from ’strongly
disagree’ to ’strongly agree’. This will provide quantitative data that can be analyzed
alongside qualitative insights from debriefing sessions. These sessions will give users
the opportunity to elaborate on their experience, discussing factors such as comfort,
immersion and any discrepancies between sensory inputs.
38
CHAPTER 4
Implementation
This chapter explains the implementation of the "Phantastic Sound Experience" based
on the design described in the chapter before. It details the systematic translation of the
design concept into an immersive virtual environment. It incorporates the integration of
the physical device, including the deployment of vibration modules, and precise creation
of dynamic sound landscapes for virtual reality. The requirements specified in Chapter
3.1 were closely studied to ensure strict alignment between the conceptual framework
and practical implementation. The first part of this chapter describes the process of
developing all basic components, namely the creation of 3D assets, sound and sensory
feedback. The second part addresses their integration in the virtual world.
4.1 Virtual environment
The virtual world forms the core of the "Phantastic Sound Experience", providing a
visually engaging and interactive environment that enables the sensory feedback systems.
As discussed in Section 3.3.1 of the design chapter, the inspiration for the virtual
environment comes from the architectural style of Ricardo Bofill, particularly his work
on Muralla Roja. The selection of this design was required in order to provide geometric
precision and minimalism to guarantee that the space has considerable aesthetic impact
while remaining unobtrusive to the user. This approach supports the user’s ability to
immerse themselves in the environment rather than distracting them from the primary
experience. In order to achieve this, all required elements had to be modeled, textured
and composed in Blender, a open source 3D software that was chosen for the project. The
primary structure of the virtual world is placed in an ethereal sky, disconnected from the
ground to create a surreal, otherworldly environment. This design choice aims to enhance
the sense of detachment from reality, which is key to deepening the user’s immersion, as
discussed in the Virtual Environment Design section. The central architectural feature
39
4. Implementation
is a cubic structure with rounded windows and rooftop outlets, reminiscent of Bofill’s
geometric and minimalistic designs that is shown in figure 4.1.
Figure 4.1: Final view in Unity
Within this environment, users are enclosed in a glass cube, which serves as their vantage
point. The cube is both a physical boundary and a symbolic barrier, isolating the user
from the dynamic external world while still allowing them to observe it closely. The
glass walls provide transparency, while the stone foundation adds weight and realism to
the structure, ensuring that it responds believably to physical interactions with other
elements, such as the spheres. The spatial layout of the environment encourages passive
interaction with dynamic objects, such as spheres that move within the space. These
spheres are not only visually engaging but also functionally important, as they respond
to audio cues and form a critical part of the dynamic feedback system. Their constant
motion and interaction with the environment add to the evolving nature of the experience,
offering users an ever-changing visual landscape to explore from their secure position
within the cube.
4.1.1 Asset creation
The creation of the virtual environment’s assets was done using Blender 4.0, chosen for
its flexibility and advanced modeling tools, as well as its open-source nature. The use
of Blender allowed for detailed modeling of both architectural structures and dynamic
objects, ensuring that the virtual world maintained high visual fidelity while optimizing
for performance in Unity, the game engine used for the implementation (discussed further
in Section 4.4. The glass cube, staircases, circular platforms, and the spherical objects
were created to not only reflect the minimalist aesthetic but also support the dynamic
interactions that take place during the experience. The glass cube was a particular focus
during the modeling phase, as it needed to convey both transparency and solidity. Its
upper surface was modeled as clear glass, while the foundation was constructed with
stone textures to ensure that it behaved realistically when interacting with the spheres.
In Blender, the physics simulation features were employed to test the cube’s response
40
4.1. Virtual environment
to these dynamic elements before exporting the models to Unity. Each element of the
virtual environment serves both a functional and aesthetic purpose, contributing to the
overall immersive experience. The spheres, which are central to the dynamic nature of
the virtual world, were modeled to move freely within the space, reacting to both the
physical properties of the environment and the audio cues that govern their behavior.
The spheres are designed to introduce variability, their movement providing an evolving
visual stimulus that enhances user engagement. The architectural elements, including
the staircases and circular platforms, were designed not only to reflect Bofill’s geometric
elegance but also to function within the dynamic environment. The staircases guide
the movement of the spheres, while the platforms provide visual balance and structure
to the floating, surreal space. Each component was measured and constructed to align
seamlessly with the interactions programmed in Unity, ensuring smooth animations and
consistent user experiences.
Figure 4.2: Rendering of the architecture in blender
4.1.2 Textures and Materials
Texturing played a critical role in bringing the virtual world to life. To maintain the
minimalist aesthetic, the textures were kept simple but detailed enough to add depth
and realism. The glass surfaces of the cube were textured to reflect light subtly, adding
to the feeling of transparency and fragility, while the stone base was textured to provide
contrast and weight. The textures were UV-mapped in Blender to ensure that they scaled
appropriately when integrated into Unity, maintaining their realism without introducing
visual artifacts.
41
4. Implementation
For the architectural elements like the platforms and staircases, soft gradients and muted
colors were used to reinforce the sense of tranquility, drawing attention away from the
architecture and toward the dynamic elements such as the spheres and audio-responsive
interactions. This texturing choice complements the audio and tactile feedback, allowing
the visual elements to support the overall immersive experience without overwhelming
the sensory inputs. The textures were prepared in blender and later rebuilt in unity.
4.2 Audio
The sound design principles of this project focus on creating a realistic audio environment
that complements the visual and tactile elements of the virtual world. With advanced
spatial audio technologies, the system ensures that sound reflects the acoustics of the
real world and aims to improve the user’s sense of presence as stated earlier in 2.3.1. The
Steam Audio plugin plays a crucial role in spatializing sound in this virtual environment.
With the help of binaural audio technologies, the scripts provide rotation and positioning
tracking, enabling low-latency, physics-based audio that supports a wide range of VR
hardware. This creates an immersive listening experience that adapts seamlessly to the
user’s movements and interactions. The soundscape can be separated into two categories:
the ambient audio landscape and the sound effects. In the next section the creation of
the ambient music is explained.
4.2.1 Conception
The concept of the soundscape was inspired by the space movie genre, with the objective
of evoking the sensation of floating in an infinite atmosphere. This reflects the virtual
environment’s geometric and surreal architectural style, which removes the user from
the physical resemblance of the real world. The sound is designed in such a way as to
reinforce this sense of detachment, transporting users into a limitless, otherworldly space,
thus enhancing both the aesthetic and emotional impact of the environment.
The central ambient track serves as the foundation of the soundscape and is constructed
upon a 4/4 beat utilizing higher frequencies. This was crucial for future frequency
separation. It was implemented in order to guarantee that the ambient sound does not
interfere with the sound effects, which are primarily located in the lower frequency range
(below 80Hz). This separation enables the sound effects, such as impacts and collisions,
which will be discussed in greater detail in the subsequent section on sound effects, to be
prominent without being overwhelmed by the ambient music, thus ensuring a clear and
balanced auditory experience.
In order to achieve this, the ambient soundscape was composed using a combination of
synthesized and pre-made sounds. These sounds were layered and mixed in Ableton Live,
which offers a variety of professional tools for sound synthesis and composition. The
combination of soft, sweeping pads evokes the vastness of space, while melodic elements
subtly guide the emotional journey of the user, encouraging a sense of curiosity and
42
4.2. Audio
peaceful immersion. This layering technique aligns with the virtual world’s design goals,
helping to maintain a cohesive mood throughout the experience and ensuring that the
audio reinforces the visual cues and the user’s interaction with the environment.
4.2.2 Sound effects
In addition to the ambient soundscape, a variety of sound effects were created with the
objective of enhancing the realism of interactions within the virtual environment. These
serve as auditory feedback in the virtual world. The aforementioned sound effects were
produced using the Foley technique, which is commonly employed in the production of
films and video games. The objective of this technique is to create realistic, synchronized
sounds that align with the on-screen actions. Foley work entails the manual recording of
sounds utilizing everyday objects, which are then adapted in a creative manner in order
to simulate the requisite audio for the given project.
The objective was to attain a high level of realism in the planning and execution of these
sound effects. To illustrate, the sound of spheres colliding with glass was created by
recording the impact of a tennis ball against a polymethylmethacrylate panel, which
mimics the acoustic properties of glass. This approach guaranteed that the auditory
representation of the collision was genuine and consistent with the physical interactions
within the virtual environment. Similarly, the collision of spheres with concrete was
replicated by dropping heavy objects onto concrete surfaces, while the dragging of con-
Figure 4.3: Layering of the ambient track in ableton
43
4. Implementation
crete was simulated using two concrete blocks. Subsequently, the recorded sound effects
were imported into Ableton Live for editing and preparation for integration into the
game engine. The editing process entailed a series of pivotal steps to guarantee that the
sound effects were not only realistic but also aligned with the audio design parameters
for the project. A principal objective was to restrict the frequency range of these effects
to below 80 Hz, a crucial factor in maintaining the clarity of the ambient soundscape,
which is predominantly comprised of higher frequencies. This separation also ensured
that low-frequency sounds could be accurately translated into tactile feedback, thereby
enhancing the physical sensations experienced by the user.
In Ableton Live, a variety of audio editing techniques were employed, including equaliza-
tion, compression, and reverb.
• Equalization was a crucial technique for the isolation of low frequencies, which were
integral to the functionality of the tactile feedback system. The use of the Audio Effect
EQ Eight with the low-cut function turned out to be sufficient to cut out undesired
frequencies
•Compression was employed to regulate the dynamic range of the audio, thus ensuring
a uniform volume and impact despite the inherent variability in the original recordings.
•Reverb was employed to align the virtual environment’s acoustics with those of the
original recordings, thus ensuring that the sound effects appeared natural and integrated
seamlessly into the space.
Figure 4.4: Example of effect use in Ableton live
These techniques employed during the audio production phase guaranteed that both the
ambient music and sound effects were harmoniously integrated to enhance the immersive
quality of the virtual world. This alignment of auditory feedback with visual and tactile
elements served to reinforce the overall experience.
4.3 Physical setup and tactile feedback
This section outlines the design and construction of the physical setup, focusing on
how the system was optimised to provide precise and synchronized haptic feedback to
the user. It explores the connection between the tactile system and the virtual world,
emphasizing the importance of synchronizing audio and vibration to maintain immersion.
44
4.3. Physical setup and tactile feedback
The chapter also discusses the technical challenges encountered during implementation
- such as vibration isolation and signal transformation - and how these were overcome
to ensure consistent, responsive performance. This discussion lays the groundwork for
understanding how the physical components support and enrich the virtual experience,
building on the previous chapters that detailed the sensory elements of the virtual world.
4.3.1 Construction of the physical component
The design of the physical setup was focused on delivering precise and effective tactile
feedback that enhances the user’s immersion in the virtual world. The unit is built on a
sturdy wooden panel mounted on a flexible metal table frame. This frame serves two key
purposes: providing structural support and isolating vibrations. Isolating the vibrations
ensures that they are concentrated on the user, preventing them from dispersing into the
surrounding environment, which would otherwise diminish the effectiveness of the tactile
feedback. The wooden panel was chosen for its durability and responsiveness, serving
as the primary surface for translating audio signals into physical vibrations. A chair is
securely fixed to the panel, offering the participant a stable seating position. The chair is
anchored with robust screws and bolts to withstand the combined weight and vibrations
produced during the experience, ensuring both comfort and stability for the user. The
tactile feedback is driven by a vibration motor mounted beneath the wooden panel. The
motor’s central positioning ensures even distribution of vibrations, providing a consistent
haptic experience regardless of where the participant is seated. This motor is connected
to a Fischer amplifier, which processes the audio signals from the computer and converts
them into the electrical signals needed to drive the motor.
The frame was built first, ensuring it was level and stable, followed by the mounting
of the wooden panel using bolts and brackets to guarantee a firm connection. Once
the panel was in place, the chair was fixed to the panel, and the vibration motor was
installed. After construction, all connections between the motor, amplifier, and computer
were tested to ensure safe, reliable operation.
4.3.2 The connection to the virtual world
The tactile feedback system was integrated with the virtual world through a direct
connection between the computer’s audio output and the Fischer amplifier. This ensures
that the physical vibrations are synchronized with the audio cues from the virtual environ-
ment, which is essential for maintaining the immersive experience. The implementation
follows the concept of the feedback loop illustrated in 3.2.3. An audio cable transmits
real-time audio signals from the virtual environment to the amplifier, which processes
them and drives the vibration motor accordingly. This real-time feedback loop ensures
that every sound the user hears in the virtual space is accompanied by a corresponding
tactile sensation, helping to solidify the perception of the virtual environment as real
and tangible. A critical aspect of the system is low latency, the delay between producing
a sound in the virtual world and the user perceiving the corresponding vibration. High
45
4. Implementation
Figure 4.5: Construction of the physical component, adding the vibration motor
latency could break immersion, as the user might sense a disconnection between the
sound and the tactile feedback. To avoid this, the system is designed to deliver vibrations
almost instantaneously in response to audio cues, ensuring seamless synchronization.
4.3.3 Synchronization and performance
Achieving synchronization between the audio and tactile feedback was one of the primary
challenges during implementation. Several strategies were employed to ensure smooth
operation:
Real-time Audio Processing: The system processes audio in real time, ensuring that
any sound produced in the virtual environment is immediately translated into vibrations.
The Fischer amplifier was fine-tuned to minimize processing delays, ensuring that the
tactile feedback was closely aligned with the audio cues.
Calibration and Testing: A comprehensive calibration process was conducted to
ensure that the intensity and timing of the vibrations were aligned with the audio cues.
Adjustments were made to the gain levels on the amplifier and the positioning of the
vibration motor to ensure that the vibrations felt natural and were perceived in harmony
with the audio.
Iterative Refinement: The tactile feedback system underwent several rounds of user
testing and refinement. User input was invaluable in identifying discrepancies between the
46
4.3. Physical setup and tactile feedback
Figure 4.6: Finished construction
audio and tactile sensations, leading to further adjustments that ensured the vibrations
were both synchronized and intuitively perceived as part of the immersive experience.
4.3.4 Challenges and solutions in physical implementation
During the construction and integration of the tactile feedback system, several key
challenges were addressed:
Vibration Isolation: It was crucial to prevent vibrations from dissipating through
the floor or neighboring structures. This was resolved by using a flexible metal frame
specifically designed to isolate and concentrate vibrations on the user.
Power Management: Ensuring consistent tactile feedback without power fluctuations
or overheating was another challenge. The Fischer amplifier was selected for its reliability
and capacity to handle real-time audio-to-tactile conversion without performance drops.
User Comfort: Delivering strong tactile feedback while maintaining user comfort was
essential. The system was designed to distribute vibrations evenly across the wooden
panel, preventing discomfort or fatigue during extended use. The placement of the chair
and the motor were optimized to ensure that the vibrations felt natural without becoming
47
4. Implementation
overwhelming.
By overcoming these challenges, the "Phantastic Sound Experience" successfully integrates
the physical component with the virtual world, creating an immersive interaction where
sound and touch work together to enhance the user’s sense of presence. In order to
enable the evaluation of sensory feedback, it is first necessary to bring together the
topics that have been discussed in this chapter thus far. The next section explores the
implementation of the virtual world in VR, detailing how audio and sensory feedback
were integrated to enhance the immersive experience.
4.4 Implementation of the experience in Unity
The most important step was to combine all the previous parts into one interactive
experience. The next phase of the development process involved importing the created
models into Unity 2022 LTS, the chosen game engine for the project. Unity was chosen as
the game engine for the project because of its extensive virtual reality support, flexibility
and robust community resources. The Steam VR package was integrated into Unity to
provide the necessary basic VR functionality and ensure compatibility with the chosen
hardware, the Vive Pro 2. The Universal Render Pipeline (URP) was used to optimize
performance and visual quality. The URP enables the creation of highly efficient render
pipelines that are tailored to the needs of the VR environment. This approach allows
the delivery of high fidelity visuals while maintaining the performance levels required for
real-time rendering. A key visual component of the virtual environment is the sky, which
is represented by a high dynamic range imaging sky map. The sky map has two functions:
it represents the environment and is the primary source of lighting. The lighting is soft
and immersive, enhancing the visual atmosphere of the virtual space. The architectural
structure created in Blender was imported into Unity as an FBX file, ensuring that
all details and textures were preserved during the transition. Subtle lighting was then
configured and baked into Unity to improve performance. The lighting baking process is
used to pre-calculate lighting interactions, reducing the computational load at runtime
and ensuring that the virtual world remains responsive and fluid.
4.4.1 Key systems
To facilitate interactivity within the "Phantastic Sound Experience", several systems
were implemented to handle dynamic object behavior and audio integration. Two core
components of this system are the Audio Manager and the Sphere Generator, both
working together to create an evolving, responsive environment.
Audio Manager: This component is important for the rendering of sound within the
environment, it is responsible for loading, playing and analysing audio. The audio track
playing in the background is subjected to analysis using Fast Fourier Transform (FFT)
functionality, which facilitates the decomposition of audio into its constituent frequency
components. The Audio Manager is able to detect peaks in the audio signal, which it then
48
4.4. Implementation of the experience in Unity
uses to trigger events within the virtual world. These events for example are responsible
for triggering the sphere generator. They also trigger other scripts that are responsible for
the movement of colliders. It also oversees the management of audio sources, including
the ambient track that serves as the foundation for the entire experience.
Sphere Generator: The Sphere Generator is the component responsible for creating
and managing the spheres within the environment. In response to audio peaks, the
Sphere Generator creates new spheres based on data received from the Audio Manager,
providing a dynamic link between the visual activity in the room and the music. Each
sphere is assigned physical properties that determine how it interacts with other objects
in the environment.
Rigid Body System: The Rigid Body System is a core component of Unity’s built-in
physics engine and plays a crucial role in the simulation of physical interactions. By
assigning a RigidBody component to each sphere, the system allows the spheres to behave
according to realistic physical principles such as gravity, mass and velocity. Physical
materials are used to define specific properties such as bounce, friction and mass, which
determine how the spheres react to collisions and interactions within the environment.
For example, the degree of elasticity controls how much the spheres bounce off surfaces,
while friction affects how they slide or roll when in contact with other objects. To
enable collisions, colliders have been strategically placed on all relevant elements within
the virtual environment. This ensures that the spheres do not pass through walls or
other objects, maintaining the illusion of physical space. Different types of colliders
- such as box colliders for flat surfaces and mesh colliders for more complex shapes -
helped define these boundaries and ensured accurate and realistic collision detection. In
addition, Unity’s physics system allows for dynamic object interaction, meaning that as
spheres collide with each other or with the environment, the physics engine calculates
the responses to these collisions in real time, including the momentum and trajectory of
the sphere after impact. This interaction further enhances the realism and immersion of
the virtual world, contributing to a more engaging and believable experience for users.
Seamless Integration for Immersion: These systems work together to create an
immersive, dynamic environment. As the ambient track plays, the Audio Manager
continually analyzes the music, triggering the Sphere Generator to fill the space with
spheres that interact with the environment and the glass cube housing the participant.
As these spheres collide with the cube, they apply pressure that causes subtle movements,
adding to the realism of the user’s experience. The dimensions of the room and the
sphere sizes were adjusted through iterative testing to ensure the space fills over the
five-minute duration of the ambient track, offering users an immersive and constantly
evolving experience.
4.4.2 Audio in unity
Each component of the virtual environment that generates audio is equipped with a Steam
Audio Source script. The script ensures that the sound is distributed correctly, taking into
49
4. Implementation
account factors such as position, air absorption and occlusion. By replicating the physical
distribution of sound through the audio source scripts, sounds can be emitted from
specific locations and resonate as they would in the physical world. The incorporation of
Steam Audio not only enhances the realism of the virtual environment, but also ensures
that audio feedback is accurately aligned with the user’s position and interaction. The
spatial audio technologies used provide users with a highly immersive experience, with
sounds triggered by virtual events resonating from appropriate locations to create a
coherent and convincing soundscape.
4.4.3 Setup of Steam Audio
Integrating Steam Audio into Unity was critical to creating an immersive audio experience.
The process began with the Steam Audio package, a tool set designed for spatial audio
in virtual environments. Its advanced features made it an appropriate choice for this
project.
The following section provides an overview of the steps required to integrate Steam Audio
into a Unity project.
1. The installation process is as follows: The first stage of the process involved downloading
and importing the Steam Audio package into the Unity project. This was achieved by
using Unity’s Package Manager to ensure that all the necessary assets and scripts were
available for integration into the virtual environment.
2. The next step is to configure the Steam Audio Manager. Following the installation
process, a Steam Audio Manager object has been created within the Unity scene. The
Steam Audio Manager is responsible for global audio settings, including the handling of
sound propagation and occlusion. The Steam Audio Manager was configured to meet the
specific requirements of the project, including modifying the speed of propagation and
creating an environment conducive to accurate reverberation and reflections.
3. Appending Steam audio source scripts: Next, a Steam Audio Source script was applied
to each audio-emitting object in the virtual environment. This script allows for accurate
spatialization of sound, taking into account the 3D position of the sound source in relation
to the user. Key settings such as distance attenuation, air absorption and occlusion were
calibrated to ensure that the sound behaved as realistically as possible.
4.4.4 Configuration
To create a cohesive audio environment, specific sounds were placed and configured in
the Unity project to facilitate the desired listening experience. The ambient track, which
serves as the foundation for the audio experience, was assigned to the Audio Manager
object. The object was placed at the center of the virtual environment to ensure an
immersive and comprehensive listening experience. Placing the ambient track in this way
ensures that the sound is evenly distributed throughout the room, creating an immersive
experience that evokes a sense of curiosity and tranquility. In terms of sound effects, each
50
4.4. Implementation of the experience in Unity
was assigned to the appropriate object within the scene. For example, collision sounds
were assigned to the spheres as they moved and interacted within the virtual environment.
The Steam Audio Source script attached to these objects ensures that the collision sounds
are not only triggered at the right moment, but are also spatially localized according to
the location of the collision relative to the user’s position. The precise placement and
configuration of the sounds contributes significantly to the realism and immersion of the
experience. Managing multiple audio sources in a virtual environment requires planning
to avoid overload and ensure clarity. It was important to provide the right mix of sounds
such as collisions while maintaining the ambient track. Steam Audio allowed precise
calibration of each sound source, ensuring that users hear relevant sounds based on their
movements.
4.4.5 Optimizing for integration and performance
Given the real-time nature of VR, it was important to optimize audio performance to
maintain a seamless and immersive experience. Steam Audio’s real-time processing
capabilities were used to ensure minimal latency in sound propagation and spatialization.
Resource management techniques, such as eliminating distant sound sources or modifying
sound quality based on the user’s distance from the source, were also used to maintain
optimal performance without compromising audio quality. Steam Audio’s advanced
features, including occlusion and sound propagation, were instrumental in achieving
realism. The occlusion feature was used to simulate the way sound is blocked or absorbed
by surrounding objects, adding depth and realism to the audio experience. For example,
the sound of a bullet colliding with a surface can be partially muffled if another object is
in the way, as would happen in the real world. Environmental effects such as reverb have
also been carefully implemented to simulate the acoustics of different materials, ensuring
that each room in the virtual environment has a distinctive acoustic signature. It was
important to integrate the audio with other sensory systems, particularly tactile feedback,
to create a unified and coherent experience. To illustrate, when the user collides with the
glass cube, they not only hear the impact, but also experience a corresponding vibration
through the synchronized haptic feedback system. To achieve this, the every component
had to be optimized for performance to avoid any lag or disruption.
4.4.6 First iteration
The first iteration of the "Phantastic Sound Experience" marks the initial implementation
of the project, integrating all sensory feedback mechanisms - visual, audio and tactile.
This version will serve as the basis for the following user testing, which will assess the
effectiveness of the immersive experience. While the core systems are in place, further
fine tuning is required to optimize the sensory feedback and improve overall performance.
With the first iteration of the "Phantastic Sound Experience" complete, the next step was
to validate and refine the core systems - visual, audio and tactile feedback - through user
testing. While the technical components had been successfully integrated and optimised,
real-world testing was essential to assess how these systems worked together to create an
51
4. Implementation
immersive experience for participants. The user testing phase provided an opportunity
to evaluate the effectiveness of the design, gather qualitative and quantitative feedback
and identify areas for improvement. By systematically reviewing how users interacted
with the virtual world, the iterative process aimed to fine-tune the system and improve
both its performance and overall immersion. The next chapter will focus on conducting
user tests and iterating on the design based on the feedback gathered to improve the
immersion and interactivity of the system.
4.5 User tests
User testing of the "Phantastic Sound Experience" was conducted over a two-day period
with two participants who had a moderate level of experience with virtual reality. The
primary objective of these tests was to evaluate the effectiveness of the system design
in order to gain insight into how to best calibrate the system. Key topics included the
physical, auditory and tactile components of creating an immersive and engaging virtual
environment. The tests aimed to identify areas of strength and potential for improvement
by systematically varying key parameters across sessions.
Each participant interacted with the virtual environment for five minutes during three
different sessions. The first session presented the baseline, the second and third session
were with improved parameters after the given feedback. Between each session, specific
parameters were adjusted to explore their impact on the overall experience. Parameters
examined included the number of spheres generated, the physical interactions between
the spheres and the glass cube (including the movement and weight of the cube), and
the intensity of both the sound and corresponding vibrations generated by the haptic
feedback system.
4.5.1 Key Findings and analysis
The user tests provided valuable insights into how different elements of the virtual
experience impacted participants’ immersion and enjoyment, while also highlighting areas
for improvement. The key conclusions drawn from the feedback are as follows:
General experience and pacing: Participants responded positively to the overall
pacing of the experience, particularly the gradual filling of the room with spheres over
five minutes. The pacing was considered well-balanced, maintaining engagement without
becoming monotonous. The accumulation of spheres provided a dynamic interaction,
which participants found kept their attention throughout the session. However, partici-
pants also noted that while the progression was engaging, the virtual environment lacked
interactive components, leading to a sense of constraint. They expressed a desire for a
more complex and varied setting that would increase immersion and interaction.
Physical Interaction and cube movement: While the physical interaction between
the spheres and the glass cube was initially appreciated, the force of the collisions was
found to be excessive. Participants reported that the exaggerated movement of the cube
52
4.5. User tests
reduced the realism, detracting from the sense of immersion. The movement was too
significant to align with the otherwise controlled and stable environment, breaking the
illusion of the experience. Both participants suggested that adjusting the impact forces
could help create a more believable and immersive interaction.
Sound distribution and intensity: The sound distribution within the virtual space
was generally well-received, as it created a cohesive auditory environment. However,
participants noted that the intensity of certain sound effects, particularly those triggered
by collisions, was overwhelming. Excessive volume occasionally disrupted immersion
and made it difficult to focus on other sensory inputs. In addition, the sounds produced
by the colliding spheres were perceived as confusing and overly repetitive. Participants
described the auditory clutter as detracting from the experience, particularly when trying
to distinguish between meaningful interactions, such as impacts on the glass cube, and
less relevant events. Furthermore, the short duration of impact sounds was found to
reduce realism, as participants expected these sounds, especially those involving the cube,
to resonate for a longer period.
Vibration feedback and tactile response: The tactile feedback provided by the
vibration motor was perceived as too weak, limiting its ability to fully enhance the
immersive experience. Participants appreciated the added dimension that vibrations
brought to the interaction but felt that the feedback lacked intensity, diminishing its
overall effect. Both participants suggested that stronger and more calibrated vibrations
could significantly improve immersion by better aligning with the visual and auditory
elements. The mild vibrations did not adequately convey the physicality of the virtual
interactions, and more pronounced tactile feedback was recommended to better simulate
the environment.
Conclusions and recommendations for improvement: Overall, the experience was
positively received, with participants praising the concept and the integration of multiple
sensory modalities. However, several areas for improvement were identified, particularly
the need for better synchronization and calibration of feedback systems. Adjusting the
intensity of the cube’s movement, refining the sound design for clarity and realism, and
enhancing the tactile feedback were key suggestions for future iterations. Additionally,
participants recommended increasing the complexity of the environment by incorporating
more interactive elements to foster deeper engagement. Addressing these points would
help in creating a more cohesive, immersive, and satisfying virtual experience.
4.5.2 Resulting changes based on feedback
In response to user feedback, several key modifications were made to enhance the virtual
experience and address issues raised during testing. These improvements aimed to create
a more immersive and balanced experience by refining the environment, physics, audio,
and tactile feedback systems.
First, the environment was enriched with additional elements such as statues and decora-
tive objects to increase visual variety and interactivity, which can be seen in 4.7. These
53
4. Implementation
additions not only enhanced the aesthetic appeal but also contributed to a more complex
and engaging experience, addressing participants’ feedback about the simplicity of the
original setup.
Figure 4.7: View of the added statues and the glass cube the participants sit in
To further enhance interactivity, the physics of one-third of the spheres were modified
to float independently of gravity. This adjustment introduced new dimensions of move-
ment, making the virtual world more dynamic and visually captivating. These floating
spheres offered participants a varied and visually appealing interaction, improving overall
immersion.
The auditory environment also underwent significant refinement. Sphere-to-sphere
collision sounds were removed to eliminate auditory clutter, focusing the soundscape
on more meaningful interactions, such as the collisions of spheres with the glass cube.
Additionally, the impact sounds of the cube were adjusted to trigger only when the cube
was in motion, reducing unnecessary audio triggers and enhancing the clarity of sound
cues. This focused sound design helped create a more immersive and cohesive auditory
experience.
In terms of tactile feedback, the intensity of the vibrations was increased by adjusting
the amplifier gain, leading to a more pronounced and perceptible haptic experience. This
enhancement ensured better synchronization between the visual, auditory, and physical
elements, contributing to a more immersive and tangible interaction with the virtual
environment.
These changes collectively aimed to create a more immersive and engaging virtual reality
experience, addressing key feedback points from the user tests while retaining elements
that were positively received. By improving the visual, audio, and tactile components,
54
4.6. Improved system
the system now offers a more balanced and captivating experience that aligns with the
project’s goals of enhancing immersion through sensory feedback.
4.6 Improved system
The refined "Phantastic Sound Experience" represents a cohesive and well-integrated
system that blends visual, auditory, and tactile elements to create an immersive virtual
reality environment. The modifications based on user feedback, such as adding dynamic
visual elements and enhancing tactile feedback, have significantly improved the system’s
performance and user experience. Visually, the environment strikes a balance between
realism and simplicity, allowing users to engage without being overwhelmed. The
introduction of dynamic, interactive elements like floating spheres and statues increases
the system’s engagement factor. These elements provide visual variety while maintaining
a clean, structured design.
The audio system, powered by the Steam Audio plug-in, delivers a realistic 3D soundscape
that enhances user immersion by accurately tracking and adapting to movements within
the environment. Binaural audio techniques ensure precise sound spatialization, and the
refined sound cues align perfectly with both visual and tactile elements. Tactile feedback,
driven by a calibrated vibration motor, plays a crucial role in enhancing the user’s sense
of presence. The vibrations are now stronger and more synchronized with the audio
and visual elements, offering a more cohesive multisensory experience. This real-time
translation of audio signals into physical vibrations strengthens the user’s connection to
the virtual world, contributing to a more believable and immersive experience.
(a) view after 30 seconds (b) view after 120 seconds (c) view after 240 seconds
Figure 4.8: Views of the final simulation
In conclusion, the improved "Phantastic Sound Experience" presents a more balanced
virtual world for evaluating the impact of sensory feedback on user immersion and
presence. By integrating visual, auditory, and tactile feedback, the system offers a
robust platform for further research into the role of multisensory experiences in virtual
environments. How this platform was used to evaluate the impact of the sensory feedback
will be discussed in the next chapter.
55

CHAPTER 5
Evaluation
This chapter presents the evaluation including data collection, results and discussion.
The aim of this evaluation was to assess the effectiveness and usability of the Phantastic
Sound Experience system, focusing on key metrics such as user engagement, presence,
immersion and overall satisfaction. The revised implementation of the system was used to
explore how the integration of 3D audio and tactile feedback enhances the user experience
in virtual reality. Both quantitative and qualitative methods are used to evaluate the
performance of the system in relation to its design goals. Special emphasis will be
placed on the impact of multimodal sensory input - including visual, auditory and tactile
feedback - and its role in enhancing immersion, presence and interaction. By analysing
the collected data and comparing the results with existing research, insights will be
gained into how well the system achieves its objectives and contributes to a broader
understanding of sensory feedback in VR.
5.1 Setup
The set-up for evaluating the "Phantastic Sound Experience" was designed to ensure that
both the technical and participant conditions were optimised for testing the immersive
qualities of the system. The evaluation had two main components: the technical setup,
which focused on providing high-quality audio and tactile feedback, and the participant
setup, which ensured a diverse and prepared group of individuals.
The technical setup used state-of-the-art hardware, including the Vive Pro 2 headset for
high-resolution visual and spatial audio feedback, and a custom-built haptic system for
tactile feedback. A controlled environment ensured that participants could fully engage
with the virtual environment without external distractions. On the participant side, care
was taken to select individuals with varying levels of familiarity with VR technology.
By combining state-of-the-art technical configurations with a well-balanced participant
57
5. Evaluation
pool, the setup aimed to provide reliable data on how effectively the "Phantastic Sound
Experience" met its objectives in terms of immersion, presence and overall user satisfaction.
5.1.1 Technical setup
The technical setup for the Phantastic Sound Experience evaluation was designed to
provide ideal conditions for testing the immersive qualities of the system, particularly its
audio and haptic feedback components. The primary hardware used for the evaluation
was the Vive Pro 2 headset, chosen for its high-resolution display and advanced binaural
audio capabilities. The headset was integrated with Steam Audio to provide spatial audio,
allowing users to experience realistic 3D soundscapes that responded to their movement
and interaction within the virtual environment. To provide haptic feedback, a vibration
motor was mounted under a custom-built chair attached to a wooden panel. The motor
was powered by a Fischer amplifier, which ensured that low frequency vibrations were
synchronized with the audio cues. This setup provided users with a consistent and
responsive tactile experience, enhancing their sense of presence and interaction within the
virtual world. The evaluation was conducted in a empty room without noise interference.
This controlled environment ensured that participants were fully immersed in the VR
experience without external distractions. The room audio configuration used binaural
audio output from the Vive Pro 2 headset. Steam Audio managed sound propagation
and occlusion, simulating real-world auditory experiences in the virtual environment.
Real-time reverb and occlusion effects were applied to ensure that the sound dynamically
adapted to the user’s movements and interactions, enhancing the realism of the experience.
5.1.2 Participant setup
The evaluation involved 12 participants between the ages of 21 and 36, selected to
represent a diverse demographic with varying levels of familiarity with Virtual Reality
technology. This age range was chosen to ensure that both experienced users and those
less familiar with immersive systems were included, providing a balanced perspective
on the usability and effectiveness of the "Phantastic Sound Experience". No previous
experience of audio engineering or VR was required, although participants were asked
about their general familiarity with VR systems and audio-based experiences. The
selection criteria aimed to avoid bias towards audio experts or novices, instead focusing
on a broad user group to ensure usability of the system across different backgrounds.
Each participant was given detailed information about the nature of the VR experience,
including any potential auditory or tactile discomfort. Informed consent was obtained
before the evaluation began. Participants were also informed that they could withdraw
at any time. Volume levels were pre-calibrated to ensure comfort, but participants were
allowed to adjust the settings if necessary. The participants were divided into two groups
- Group A and Group B - of 6 people each. This split allowed for comparative analysis,
with each group experiencing different variations of the "Phantastic Sound Experience"
to evaluate the impact of the sensory feedback.
58
5.2. Evaluation methodology
Figure 5.1: Test Run
5.2 Evaluation methodology
This section outlines the key metrics used in the evaluation, including user experience,
immersion and motion sickness, and highlights the importance of these factors based on
previous research in human-computer interaction and virtual reality. The data collection
process is based on a standardized questionnaire to ensure consistency, followed by
thorough statistical analysis to compare performance across different user groups. Finally,
the procedure for conducting the evaluation is detailed, emphasizing consistency in
59
5. Evaluation
participant setup, interaction with the virtual world, and post-session debriefing to
capture comprehensive feedback.
Through this methodology, the study aims to provide empirical evidence on how sensory
feedback affects the VR experience, contributing to both system improvements and the
wider academic discussion on immersive technologies.
5.2.1 Evaluation metrics
The evaluation focuses on key metrics to assess how effectively the "Phantastic Sound
Experience" meets its objectives. These metrics are based on established research in
human-computer interaction, presence and immersion in VR, ensuring a comprehensive
evaluation of the system’s performance and user experience.
Presence: Presence is a critical measure of how "real" the virtual environment feels. It
is assessed using the Presence Questionnaire, which assesses sensory fidelity and user
involvement. Research by Witmer and Slater [89] emphasizes the importance of presence
in VR systems, as it has a direct impact on user engagement and task performance.
Immersion: The impact of high-resolution displays and 3D audio is evaluated based
on the user’s sense of being immersed in the virtual environment. Research by Slater
and Wilbur [68] and Hendrix and Barfield [37] role of visual fidelity and spatial audio in
enhancing the immersive experience. The evaluation will focus on how effectively these
elements contribute to a seamless sense of presence.
Motion sickness: A custom questionnaire assesses physical discomfort, including
symptoms such as nausea and dizziness. Motion sickness is a critical factor in VR
usability, and understanding its impact helps ensure the system is comfortable for
extended use.
Emotional Response: Participants’ emotional responses (e.g. excitement, fear, enjoy-
ment) are tracked to assess the affective impact of the VR experience. This metric helps
to measure the emotional connection users form with the virtual environment, which is
essential for measuring engagement and immersion.
Effectiveness of sensory feedback: The effectiveness of the sensory modalities is
measured by how they enhance the overall VR experience. Tactile feedback will be
evaluated for its contribution to haptics and realism, while 3D sound will be evaluated
for its ability to create spatial atmosphere, improving sound localization and overall
engagement.
By focusing on these areas, the evaluation aims to provide a comprehensive understanding
of the system’s strengths and areas for improvement. These metrics not only assess
the performance of the VR system, but also contribute to a broader understanding of
how sensory feedback influences user engagement, immersion and presence in virtual
environments.
60
5.2. Evaluation methodology
5.2.2 Data collection
The data collected for this evaluation is based on a questionnaire adapted from the
Presence Questionnaire (PQ) and the Immersive Tendencies Questionnaire (ITQ), which
were developed by Witmer and Singer [90]. These instruments, which have gained
considerable recognition in the field, have been designed to assess the extent of presence
and immersion in virtual environments. The PQ is specifically designed to assess the
sense of being present, whereas the ITQ is intended to examine individual tendencies
towards becoming immersed in virtual environments. In order to assess key aspects of the
"Phantastic Sound Experience", the questionnaire was tailored accordingly. Questions as
"How aware were you of your actual physical surroundings while in the VR environment?"
and "How effectively did the virtual environment engage your senses?" were derived from
the PQ and adapted to suit the context of this study, thus maintaining a focus on sensory
fidelity and engagement as suggested by Witmer and Singer. Their work emphasizes that
presence arises from a combination of directed attention, sensory input, and environmental
immersion. The impact of sensory feedback was assessed by incorporating specific
questions for Group B, such as "To what extent did the tactile sensations align with your
visual and auditory perceptions?" and "Please rate the efficacy of the tactile feedback
in enhancing the realism of the virtual reality experience." This focus is consistent with
the concept of "sensory factors" as outlined in Witmer and Singer’s framework, the
specific questions are shown in the appendices A. This provides a solid foundation to
explore if the integration of multimodal feedback can significantly enhance the sense
of immersion and presence by engaging multiple senses in a coherent manner. The
data from both groups were subjected to descriptive statistical analysis, with metrics
such as the mean, median, and mode providing insights into the overall user experience.
A t-test was employed to conduct statistical comparisons between Group A (without
sensory feedback) and Group B (with sensory feedback), with the objective of identifying
significant differences, particularly in how users rated immersion and presence. The
results demonstrate a trend where multimodal sensory feedback was found to enhance the
sense of presence and immersion, thereby supporting the original hypotheses and aligning
with the empirical findings from Witmer and Singer’s research on sensory engagement
and virtual environments.
5.2.3 Procedure
The evaluation procedure was designed to systematically collect data on immersion,
presence, motion sickness and the effectiveness of sensory feedback in the Phantastic
Sound Experience. Each test session followed a structured format to ensure consistency
and reliability of data collection.
1. Introduction and briefing: Each session began with a 15-minute briefing where
participants were introduced to VR and informed about the specific research focus areas:
immersion, presence, motion sickness and sensory feedback. This briefing helped to
contextualise the experience for the participants and ensure that they understood the
key aspects they were evaluating.
61
5. Evaluation
2. Setup and calibration: Participants were seated and the VR system was individually
calibrated to ensure optimal visual and auditory performance. This step was critical to
ensuring a consistent experience for all users.
3. Virtual world interaction: Participants spent five minutes immersed in the
Phantastic Sound virtual world. This duration was chosen to allow sufficient time to
explore the VR environment while minimizing the risk of fatigue and over-stimulation.
4. Interview with questionnaire: Following the session, participants completed a
questionnaire designed to collect feedback on the following areas:
- Presence: Measured the extent to which participants felt present in the virtual world,
including sensory involvement and realism.
- Immersion: Assessed how engaged participants felt within the virtual environment and
the quality of the visual and auditory experience.
- Motion sickness: Evaluated any symptoms of discomfort, such as nausea or dizziness,
experienced during the VR session.
- Qualitative feedback: Participants were encouraged to provide additional comments
and suggestions for improvement.
For Group B, an additional section of the questionnaire focused on the effectiveness of
sensory feedback, in particular how tactile feedback enhanced the sense of touch and
realism.
5. Debriefing: After completing the questionnaire, participants were debriefed to
discuss their overall experience. This provided the researchers with valuable qualitative
insights and allowed for further exploration of any specific issues or suggestions for system
improvements.
5.3 Results
This section presents the results of the evaluation of the "Phantastic Sound Experience".
The results are divided into three parts: quantitative data, qualitative feedback and a
comparison with the initial hypotheses. The quantitative analysis provides statistical
insights into key user experience metrics such as immersion, presence and motion sickness,
comparing the performance of Group A (without sensory feedback) and Group B (with
sensory feedback). The qualitative results provide a deeper understanding of participants’
reactions, both positive and negative, to their experience in the virtual environment.
Finally, a comparison with the study hypotheses will assess whether the sensory feedback
system met the expectations set during the design phase. This analysis will help
determine the overall effectiveness of integrating multimodal feedback and inform future
VR developments based on the findings.
62
5.3. Results
5.3.1 Quantitative results
The quantitative data collected from the study was analyzed using descriptive statistics
such as mean, median and mode, supplemented by a t-test to compare the performance
of groups A and B. These statistical measures provide insight into central tendencies and
variability, giving a clear picture of the user experience across the two conditions.
Presence: The sense of presence, measured by the Presence Questionnaire, revealed a
difference between the two groups, with Group B showing an average mean score of 3.5
compared to Group A’s 2.92. This indicates that the multimodal feedback, particularly
the combination of 3D audio and tactile feedback, contributed to a slight increase in
the participants’ perception of being ‘present’ in the virtual environment. However, the
t-test results did not show statistical significance, meaning that the observed difference
between the groups, while present, was not strong enough to confirm a significant impact
of the sensory feedback on presence.
Figure 5.2: Summary of answered questions
Questions in 5.2
Question 1: How strongly did you feel ’present’ in the virtual environment during the
experience?
Question 2: To what extent did you feel like you were ’inside’ the virtual world rather
than merely observing it?
Question 3: How aware were you of your actual physical surroundings while in the VR
environment?
Question 4: How effectively did the virtual environment engage your senses?
63
5. Evaluation
Table 5.1: T-Test Values for Presence Questions
Question Q1 Q2 Q3 Q4
Presence 0.20 0.79 0.67 0.17
Immersion: Group B (with sensory feedback) reported a slightly higher level of immer-
sion, with an average mean score of 3.78 compared to 2.89 in Group A. This lends support
to the hypothesis that the integration of tactile and auditory feedback would enhance user
engagement. Group B participants reported a heightened sense of physical involvement
in the virtual environment, attributing this to the synchronized tactile vibrations and
sound. Furthermore, comments regarding time perception indicate an enhanced sense
of immersion. Several participants reported losing track of time or feeling absorbed
in the experience. Descriptions such as "I was fully absorbed into the virtual reality"
illustrate the engagement experienced by Group B. In contrast, Group A, which did not
receive sensory feedback, reported less immersive experiences, with participants frequently
maintaining a clear awareness of time or external surroundings. Some participants in
Group A observed that "nothing really happened" or that they were aware of sounds
from their environment, which made it more challenging to lose track of time. Although
the t-test for the third question indicated a statistically significant result, the observed
difference between groups was not sufficiently pronounced to be considered a definitive
indicator, thereby limiting the extent to which these findings can be interpreted with
certainty.
Figure 5.3: Summary of answered questions
Questions in 5.3
64
5.3. Results
Question 1: How would you rate your level of immersion while being in the virtual
environment?
Question 2: How engaging did you find the VR experience overall?
Question 3: To what degree did the virtual environment replace your real-world sensory
experiences?
Table 5.2: T-Test Values for Immersion Questions
Question Q1 Q2 Q3
Immersion 0.15 0.34 0.02
Motion sickness: The assessment of motion sickness was based on the severity of the
symptoms experienced by the participants, who rated their experience on a scale from 0
(no symptoms) to 5 (very strong symptoms). The data for Group A and Group B are
presented in the following:
Group A (without sensory feedback) exhibited a mean motion sickness rating of 2.1,
indicating moderate symptoms. The range of responses spanned from no symptoms (0)
to strong symptoms (4). Group B (with sensory feedback) demonstrated a lower mean
rating of 1.5, indicating a generally lighter experience of motion sickness. Participants in
this group exhibited fewer symptoms overall, with responses primarily in the lower range
of the scale (0-2).
Figure 5.4: Comfort and motion sickness
Emotional response: The emotional responses of participants in Groups A and B
revealed notable differences in the impact of the virtual experience on their emotional
65
5. Evaluation
states. Group B, which received sensory feedback, reported higher emotional intensity,
with an average intensity score of 3.33, in comparison to Group A, which reported an
average intensity score of 2. Participants in Group B expressed a range of emotions,
including excitement, calmness, and curiosity, with notable instances of relaxation and
happiness. Furthermore, this group reported a stronger connection to the experience,
with emotions such as surprise and excitement featuring prominently. In contrast, Group
A, which did not receive sensory feedback, reported a reduction in emotional intensity,
with calmness and curiosity being the predominant feelings. Although some participants
in Group A reported experiencing emotions such as anxiety and excitement, the overall
emotional engagement was found to be lower. This indicates that the provision of sensory
feedback in Group B significantly enhanced the emotional depth and engagement with
the virtual environment. These findings indicate that the provision of multimodal sensory
feedback can evoke more pronounced emotional responses, thereby facilitating a more
immersive and emotionally resonant experience.
Figure 5.5: Distribution of different emotions in groups
Effectiveness of sensory feedback: The effectiveness of tactile feedback in contribut-
ing to the realism of the VR experience was rated by participants of group B with a
mean score of 3.5, indicating that it was perceived as moderately effective. Similarly,
the alignment of tactile sensations with the visual and auditory elements of the virtual
environment was rated with a mean score of 3.5, indicating that the integration was
generally successful, although there was scope for further improvement. In contrast, the
contribution of the sound to engagement and immersion was rated more highly, with a
66
5.4. Analysis and interpretation
mean of 4.17, which serves to highlight its significant impact on enhancing the overall
experience. These findings indicate that while tactile feedback enhances the realism of the
experience, sound plays a more significant role in fostering immersion and engagement.
This suggests that future enhancements to tactile feedback could further improve its
effectiveness.
5.3.2 Qualitative results
The qualitative feedback from both Group A and Group B participants provided valuable
insights into the user experience, highlighting both successes and areas for improvement
in the system. Group B, who experienced the full sensory feedback, responded positively
to the immersive nature of the visual, auditory and tactile elements. Many participants
commented that the coordination between audio and tactile sensations made the virtual
world feel more tangible and engaging, transporting them into a mental space isolated
from reality. This is in line with the aims set out in the Design chapter, where the
integration of physical and auditory feedback was intended to enhance immersion.
However, some Group B participants pointed out certain sensory inconsistencies, such as
the distracting noise of the virtual spheres, which detracted from the sense of immer-
sion. There was also frustration with the lack of interactivity, particularly the inability
to manipulate objects in the environment. While the sensory feedback was effective
in enhancing the experience, these comments suggest that further refinement of the
interactivity and sound design could create a more cohesive virtual world.
In contrast, participants in Group A, who did not receive tactile or auditory feedback,
provided less detailed and enthusiastic responses. They appreciated the visual aesthetics
of the experience, but their feedback focused more on technical issues such as glitches
and motion sickness. Without the grounding effect of haptic feedback, participants were
more aware of the limitations of the system, pointing to the need for improvements in
stability and comfort.
Suggestions for improvement: In both groups, participants recommended strengthening
the coordination between the sensory elements. In Group B there were suggestions to
improve tactile feedback and make vibrations more pronounced, reduce distracting noise
and improve interactivity with virtual objects. These suggestions are consistent with
findings from previous research that emphasizes the importance of synchronizing sensory
feedback to maintain immersion (as discussed in section 5.2).
5.4 Analysis and interpretation
The analysis and interpretation section examines the performance of the "Phantastic
Sound Experience" system in relation to the hypotheses outlined at the beginning of the
study. It begins by assessing the system’s ability to meet key objectives, particularly in
terms of improving immersion, presence, user satisfaction and reducing motion sickness
through the use of multimodal sensory feedback.
67
5. Evaluation
This section also compares the results of this project with related work in the field,
positioning the "Phantastic Sound Experience" within the broader context of VR research.
By examining similarities and differences with other studies, the analysis highlights the
unique contributions and advances made by this system, particularly in the integration
of tactile and auditory feedback.
The final section considers the limitations encountered during the evaluation, such as
the small sample size and technical limitations, which may have influenced the results.
These limitations are considered alongside suggestions for refining the system, including
improvements to sensory alignment and interactivity. By reflecting on these aspects, this
section not only confirms the success of the system in achieving its primary objectives,
but also sets the stage for future improvements.
5.4.1 Comparison with hypotheses
The evaluation of the "Phantastic Sound Experience" confirms the core hypotheses
proposed at the beginning of the study. As discussed in the results, sensory feedback
played a key role in enhancing the virtual reality experience.
Sensory feedback will increase the sense of presence:
Group B showed higher presence scores, confirming the hypothesis that multimodal
feedback increases the user’s sense of being in the virtual world. Sensory fidelity, driven
by tactile and 3D audio cues, helped users feel more connected to the environment. This
supports research by Witmer and Singer [90] which shows that sensory cues significantly
improve the user’s sense of presence. The close alignment of visual, auditory and haptic
elements achieved this increase in presence, as predicted by the study design.
Sensory feedback will enhance immersion:
The data showed a significant improvement in immersion for Group B (with sensory
feedback). The integration of 3D audio and tactile feedback successfully deepened
participants’ involvement, as evidenced by higher scores on questions related to immersion.
This supports previous research emphasizing the role of sensory integration in creating a
fully immersive VR environment. The tactile elements, synchronized with audio feedback,
provided a multisensory experience that went beyond visual immersion alone. This was
in line with the aims of the project and the findings of the literature reviewed earlier, in
particular the work of Slater and Wilbur [68] which highlights that deeper immersion
correlates with richer sensory input.
Sensory feedback can reduce motion sickness:
Lower reports of motion sickness in Group B supported the hypothesis that sensory
feedback reduces discomfort. The tactile feedback likely provided participants with a
greater sense of grounding and reduced disorientation, which helped to alleviate motion
sickness. This finding is consistent with existing VR studies showing how sensory
anchoring reduces cognitive dissonance and improves user comfort.
68
5.4. Analysis and interpretation
Sensory feedback has an impact on overall user satisfaction:
Group B participants expressed higher levels of satisfaction with the VR experience,
consistent with the hypothesis that sensory feedback enhances the overall enjoyment
and effectiveness of VR systems. In particular, tactile feedback was highlighted as an
important factor in improving the realism and emotional engagement of the experience.
In summary, the performance of the system largely confirmed the hypotheses, with
sensory feedback playing a critical role in enhancing the user experience in terms of
immersion, presence, motion sickness and satisfaction.
5.4.2 Comparison with related work
The results of the Evaluation of the project align with existing research on the role of
sensory feedback in virtual reality, while contributing new insights into the specific effects
of tactile and auditory feedback on user immersion, presence, and comfort.
Immersion and presence
Previous studies, such as those by Slater and Wilbur [68],and Hendrix and Barfield [37],
emphasize the role of high visual fidelity and spatial audio in enhancing immersion and
presence in VR. This study corroborates those findings, demonstrating that Group B,
which experienced 3D audio and tactile feedback, reported significantly higher scores for
immersion and presence than Group A, which lacked sensory feedback. These results
reinforce the importance of sensory input in creating a deeply engaging virtual experience,
echoing the findings of Dinh et al. [26], who found that multimodal sensory integration
enhances the sense of presence.
multisensory integration
Research by McMahan et al. [52] highlights how tactile feedback enhances interaction
and realism in VR environments. The "Phantastic Sound Experience" supports these
conclusions, showing that Group B participants felt more connected to virtual objects
through the tactile vibrations, which added an additional layer of realism missing in
Group A. This indicates that haptic feedback, when well-synchronized with visual and
auditory stimuli, significantly improves the user’s overall experience and sense of presence.
Immersion and sensory feedback
In Eidenberger’s Virtual Jumpcube, the integration of olfactory and tactile stimuli
enhanced immersion by synchronizing non-audiovisual sensory inputs with visual cues [27].
This is consistent with the Phantastic Sound evaluation, where Group B, experiencing
haptic feedback, showed higher immersion than Group A, underscoring the importance of
coordinated sensory stimuli. On the other hand, Bruun-Pedersen et al. [15] explored the
impact of multimodal feedback on perceived exertion during VR exercise. Although their
results did not reveal significant changes in exertion, they highlighted the importance
of sensory synchronization for realism, aligning with this study’s finding that sensory
feedback improves immersion.
69
5. Evaluation
Motion sickness and physical discomfort
Group B reported fewer symptoms of motion sickness than Group A, suggesting that
grounding participants with tactile feedback helped alleviate discomfort and disorientation.
This aligns with the conclusions of Dinh et al. [26] who observed that multimodal feedback
reduces physical discomfort in VR. Both the Virtual Jumpcube and Phantastic Sound
studies demonstrated that sensory feedback reduces motion sickness. Eidenberger noted
that tactile stimuli helped minimize nausea by creating a more immersive experience [27].
Similarly, Group B participants in this study reported fewer motion sickness symptoms,
confirming the hypothesis that haptic feedback reduces disorientation. In contrast, Bruun-
Pedersen et al. focused more on exertion than motion sickness, though they acknowledged
cybersickness challenges during pilot testing, emphasizing the need for careful sensory
synchronization in reducing discomfort[15].
User experience and presence
Eidenberger’s findings indicated that olfactory and tactile feedback significantly enhanced
user excitement and enjoyment, a conclusion mirrored by the Phantastic Sound results,
where Group B participants described the experience as more immersive due to multi-
sensory engagement [27]. Bruun-Pedersen et al. found that sensory feedback increased
participants’ perception of realism in VR, even though the statistical differences were not
strong. This qualitative feedback aligns with the Phantastic Sound evaluation, where
tactile feedback enhanced the sense of being ‘inside’ the virtual environment [15].
Summary of findings
Both the Phantastic Sound and Virtual Jumpcube studies show that synchronized sensory
feedback increases immersion, reinforcing the idea that multisensory input deepens the
user’s engagement with the virtual environment. In contrast, Bruun-Pedersen et al.
found that while sensory feedback had less impact on perceived exertion, it still played a
role in shaping the overall experience. Motion sickness was another key area where the
Phantastic Sound and Virtual Jumpcube studies agreed, showing that sensory feedback
helped to reduce discomfort and nausea. However, Bruun-Pedersen et al. focused
primarily on exertional challenges rather than motion sickness. Despite these differences
in focus, all three studies agreed that sensory feedback had a positive impact on the user
experience, increasing satisfaction and presence, although the emphasis in each study
varied between immersion and physical exertion [27, 15]. This comparison underscores
the importance of integrating well-synchronized sensory feedback into VR experiences to
maximize immersion, reduce discomfort, and enhance overall user satisfaction.
5.4.3 Limitations
While the evaluation provided valuable insights, several limitations should be acknowl-
edged:
Sample size: The evaluation was conducted with a relatively small sample size (12
participants). While the results showed clear trends, the limited number of participants
70
5.5. Summary of findings
may have affected the generalizability of the findings. Future studies would benefit from
a larger, more diverse pool of participants to increase the statistical power and reliability
of the results.
Duration of interaction: Each participant spent only five minutes in the virtual world.
Although this duration was chosen to avoid overstimulation and fatigue, it may not
have been sufficient to fully capture the long-term effects of sensory feedback on user
immersion, presence, or motion sickness. Longer exposure may provide more nuanced
insights into how users adapt to and experience sensory feedback over time.
Technical limitations: The system’s reliance on specific hardware (e.g., Vive Pro
2 for VR and vibration motors for haptic feedback) may have influenced the results.
Differences in hardware configurations could alter the sensory experience, particularly
in terms of audio quality and haptic feedback intensity. Future research could explore
different hardware setups to see if these findings hold across platforms.
Limitations of interactivity: Several participants in Group B noted the lack of
interaction with virtual objects, such as the inability to manipulate or throw the virtual
spheres. This lack of interaction may have reduced overall immersion, even with the
presence of sensory feedback. Addressing these interactivity limitations in future iterations
of the system could lead to even greater engagement and a more fully immersive experience.
Trade-off between realism and performance: The visual and audio fidelity of the
system was high, but this occasionally led to performance bottlenecks that could have
affected the user experience. Although optimizations such as baking lighting were used,
some participants still reported minor lag or visual glitches. Future work should focus on
balancing high fidelity visuals with system performance to maintain immersion without
introducing technical glitches.
In conclusion, while the evaluation provided strong support for the initial hypotheses,
these limitations suggest areas for further refinement and exploration. Addressing these
limitations in future studies will help to provide a more comprehensive understanding
of the role of sensory feedback in VR and enhance the effectiveness of the "Phantastic
Sound Experience" system.
5.5 Summary of findings
The evaluation of the "Phantastic Sound Experience" provided insights into the impact of
sensory feedback, including 3D audio and tactile sensations, on user immersion, presence
and satisfaction in a VR environment. The results confirm many of the initial hypotheses,
while also highlighting areas for improvement in future iterations of the system.
Key findings from the evaluation include:
- Presence: The integration of multimodal sensory feedback, particularly tactile feedback,
was instrumental in increasing the sense of presence. Group B participants felt more
’present’ in the virtual environment, experiencing a stronger connection to the space, as
71
5. Evaluation
indicated by higher scores on both quantitative and qualitative metrics. The seamless
alignment of sound, touch and visuals helped create an immersive experience where
participants felt they were truly "inside" the virtual world.
Immersion: The presence of tactile feedback and 3D audio significantly increased the
overall level of immersion. Participants in Group B (with sensory feedback) reported
higher levels of immersion compared to Group A. Sensory cues, such as vibrations
synchronized with audio and visual elements, created a more convincing and immersive
virtual environment, enabling users to engage more deeply with the VR experience.
Motion sickness: Group B experienced lower levels of motion sickness, likely due to the
grounding effects of tactile feedback. By providing participants with physical cues that
synchronized with their virtual actions, the system minimized disorientation, leading to
a more comfortable experience. This suggests that sensory feedback may play a role in
mitigating common VR problems such as dizziness and nausea.
Sensory feedback: Group B participants rated the effectiveness of sensory feedback
highly, with particular praise for the integration of vibrations that matched visual and
auditory events. However, there were still reports of slight discrepancies between sensory
modalities, highlighting an area for refinement in future versions of the system.
User satisfaction: Overall satisfaction was higher in Group B, with participants noting
the realism and emotional engagement provided by the sensory feedback. Tactile feedback
in particular was seen as a critical element that contributed to the sense of realism and
increased user enjoyment.
Overall, the "Phantastic Sound Experience" proved to be an effective and engaging virtual
reality system, with the integration of tactile feedback and 3D audio greatly enhancing
user immersion, presence and satisfaction. These findings provide a strong foundation
for future research and improvements in multisensory VR experiences. By addressing the
minor limitations and extending the successful elements of the current system, future
iterations have the potential to provide even more immersive and compelling virtual
environments.
5.6 Discussion
The analysis of immersion levels between Group A (without sensory feedback) and Group
B (with sensory feedback) revealed some important, though modest, differences in how
participants perceived their virtual reality experience. Group B, which experienced tactile
vibrations synchronized with the audio feedback, reported slightly higher immersion,
with an average mean score of 3.78 compared to 2.89 for Group A. While this result
aligns with the hypothesis that multisensory feedback would enhance user engagement,
the statistical analysis through t-tests showed that this difference was not consistently
significant across all questions. The results of the t-test demonstrated significance in
only one of the immersion questions. The question "To what extent did the virtual
environment replace your real-world sensory experiences?" yielded a p-value of 0.02. This
72
5.6. Discussion
indicates that participants in Group B reported a greater sense of immersion in the
virtual environment with regard to the replacement of real-world sensory input, which is
likely attributable to Group A missing the auditive experience. However, no statistically
significant differences were observed in the other questions related to immersion (e.g.,
p-values of 0.15 and 0.34 for the remaining questions). This lack of statistical significance
necessitates a more cautious interpretation of the data, as it suggests that while Group
B rated the experience as slightly more immersive, the difference may not be sufficiently
pronounced to draw definitive conclusions.
Minor differences in immersion ratings:
Despite Group B consistently reporting higher scores across all immersion questions, the
actual difference in ratings was relatively minor. To illustrate, the initial inquiry regarding
immersion, "How would you rate your level of immersion in the virtual environment?,"
yielded mean scores of 4.0 for Group B and 3.33 for Group A. Similarly, the subsequent
question concerning overall engagement exhibited means of 3.67 for Group B and 3.0 for
Group A. These outcomes, while indicating that the participants in Group B perceived
greater involvement in the virtual experience, demonstrate only moderate enhancements.
The third question was as follows: "To what extent did the virtual environment replace
your actual sensory experiences in the real world?" (which yielded a significant p-value)
is the most meaningful indicator of the potential benefits of sensory feedback. This may
be attributed to the fact that the tactile vibrations facilitated a tangible connection
between the participants and the virtual world, thereby enhancing the illusion that the
VR environment had replaced their real-world surroundings. Nevertheless, the relatively
modest effect sizes observed in the remaining immersion questions suggest that, on the
whole, the sensory feedback had a somewhat limited impact on the participants’ sense of
immersion.
Insights from participant feedback:
The qualitative feedback from the participants lends support to the numerical data
to a certain extent. A significant number of Group B participants observed that the
synchronized audio and tactile vibrations enhanced their physical engagement with the
virtual environment. Some participants even reported losing track of time, with responses
such as, "I was fully absorbed into the virtual reality," and, "It could have been five
seconds or two hours." These statements demonstrate how sensory feedback can facilitate
immersion, thereby enhancing the engagement and disorientation of the experience in a
positive manner.
In contrast, participants in Group A, who did not receive sensory feedback, were more
conscious of their physical environment. Some also observed that the absence of tactile
stimulation impeded their ability to fully immerse themselves in the experience. A number
of participants observed that they remained conscious of external auditory stimuli and
the passage of time, which is likely to have diminished their overall sense of immersion.
Limitations and potential explanations:
73
5. Evaluation
Regarding these findings, the low t-test values for the majority of questions highlight the
necessity to contemplate potential constraints inherent to the study design. The study’s
statistical power is constrained by the relatively small sample size of six participants
per group. A larger sample size would provide a more robust dataset, which could
potentially reveal more significant differences between the groups. The low statistical
power in this case renders it challenging to draw robust conclusions regarding the true
impact of sensory feedback on immersion. A further potential limitation is the relatively
brief exposure to the virtual reality experience, which lasted only a few minutes. It is
possible that the effects of multisensory feedback, particularly those pertaining to the
sense of touch, may require a longer period of interaction to become fully apparent. A
longer exposure time may permit participants to become more fully immersed, which
could result in more pronounced effects on both subjective immersion and presence.
Furthermore, the calibration of sensory feedback may have been a contributing factor.
Some participants in Group B indicated that the vibrations, although present, were not
sufficiently intense to have a notable impact. This indicates that although the sensory
feedback enhanced immersion to a certain degree, its potency or timing may have been
inadequate to generate a genuinely immersive experience.
While the study provides some evidence that multisensory feedback, particularly tactile
feedback, enhances immersion, the small effect sizes and lack of statistical significance in
most questions indicate that the impact may be more modest than originally hypothesized.
The one statistically significant result suggests that sensory feedback has the potential to
replace real-world sensory input, but further research with larger sample sizes and more
pronounced feedback mechanisms is needed to fully explore this effect. Future studies
could improve the calibration of feedback and increase interaction times to allow for a
deeper investigation into the role of multisensory inputs in enhancing immersion in VR
environments.
74
CHAPTER 6
Conclusion and future work
6.1 Areas for improvement
Although the evaluation of the “Phantastic Sound Experience” was largely positive, several
key areas for improvement were identified. One of the primary issues highlighted during
the user tests was the occasional mismatch between sensory modalities, especially between
the tactile feedback and the visual or auditory cues. While the vibrations contributed
positively to immersion, inconsistencies in synchronization were noted. Participants felt
that the vibrations were either too weak or out of sync with the visual stimuli, which
reduced the overall sense of realism and presence. The quantitative results also indicated
that while Group B reported higher levels of immersion and presence, the t-tests showed
no significant difference between the two groups across most of the metrics. This suggests
that while sensory feedback improved the experience for some users, the current system’s
implementation of haptic feedback did not lead to substantial improvements across the
board. To address this, future development should focus on enhancing the precision and
timing of the feedback, as well as exploring more dynamic and responsive feedback systems
that can adjust in real-time to user interactions. Another area requiring attention is the
intensity of the tactile feedback. Participants expressed a desire for more pronounced and
varied vibrations, which would align better with the visual and auditory stimuli in the
environment. As noted in the feedback, stronger tactile sensations would help users feel
more physically present in the virtual world. By improving the haptic system to offer a
wider range of tactile responses, the overall realism and interactivity of the system could
be significantly enhanced. Finally, the small sample size (12 participants) limited the
statistical significance of the findings. Although Group B’s scores were consistently higher,
the small sample may have prevented more conclusive results. Future studies should
consider larger, more diverse participant groups to better assess the generalizability of
the findings and gain deeper insights into how sensory feedback affects different types of
users.
75
6. Conclusion and future work
6.2 Implications for future work
The findings from this study provide a foundation for further exploration of sensory
feedback in virtual reality environments. One key area for future research is the syn-
chronization between sensory modalities, particularly between audio, visual, and tactile
feedback. As participants in Group B reported that these elements were occasionally
out of sync, improving synchronization could lead to a more cohesive and immersive
experience. This could involve refining the system’s latency, ensuring that vibrations
occur precisely in tandem with visual and auditory stimuli. The emotional responses from
participants, particularly their altered sense of time, suggest another interesting avenue
for research. Group B participants, who experienced tactile feedback, often reported
losing track of time during the virtual experience, indicating deeper immersion. Future
research could explore the connection between time perception and sensory feedback,
investigating how different combinations of sensory inputs (or their absence) affect users’
sense of time within virtual environments. Moreover, further exploration of user comfort
and the potential for sensory feedback to reduce motion sickness is necessary. Although
Group B reported lower levels of motion sickness, the differences were not statistically
significant. Future studies could focus on how varying the intensity, timing, or location
of haptic feedback might improve comfort and reduce nausea during longer VR sessions.
Exploring how sensory feedback can adapt dynamically based on user responses could also
enhance comfort and immersion. Lastly, this research opens the door for more complex
interactions between sensory modalities. Future systems could experiment with adaptive
sensory feedback that responds not only to the virtual environment but also to user
input, enhancing interactivity and presence. Additionally, investigating how individual
differences—such as prior experience with VR or sensitivity to motion sickness—affect
the perception of sensory feedback will provide insights into designing more personalized
and effective VR experiences.
6.3 Conclusion
The “Phantastic Sound Experience” was designed with the primary aim of exploring
how multisensory feedback, specifically 3D audio and tactile vibrations, impacts user
immersion, presence, and satisfaction in virtual reality environments. The system
was developed to address the research questions posed at the outset, focusing on how
synchronized sensory modalities influence user experience. By allowing the participants to
engage with synchronized haptic and auditory feedback, the system created an immersive
experience that enabled the detailed evaluation of these sensory elements. The results
confirmed that sensory feedback enhances the immersive experience. Group B, which
experienced both auditory and tactile feedback, reported higher levels of immersion and
presence compared to Group A, which experienced only visual and auditory feedback.
While the increase in presence was modest, with no statistically significant difference found
in the t-test results, the qualitative feedback indicated that the synchronized feedback
contributed to a richer, more engaging experience. These findings are consistent with the
76
6.3. Conclusion
works of Eidenberger and Bruun-Pedersen, who similarly explored the role of sensory
modalities in VR but in different contexts. In comparison to Eidenberger’s study, which
highlighted olfactory and tactile integration, this project focused on auditory and tactile
feedback, reaffirming the importance of multimodal integration for deeper engagement in
virtual spaces. Bruun-Pedersen’s research, while more focused on exertion, also highlighted
how synchronized sensory feedback can increase the realism and user involvement in
virtual environments, supporting the findings of this study. The system’s design allowed
for a focused investigation into how sensory feedback can enhance user immersion and
presence. Although the differences in user-reported immersion and presence between the
groups were not substantial enough to indicate statistical significance, they suggest that
even slight enhancements in sensory feedback can positively affect the virtual experience.
Moving forward, further refinements in the intensity and precision of feedback, as well as
larger-scale studies, could better explore these subtle effects and provide more conclusive
results.
In conclusion, the “Phantastic Sound Experience” offers insights into the role of multisen-
sory feedback in virtual reality, contributing to a growing body of research in this area.
The comparisons drawn with the works of Eidenberger and Bruun-Pedersen underscore
the importance of carefully synchronizing sensory modalities to maximize immersion
and presence. The findings from this study suggest that future research should explore
even more sophisticated feedback mechanisms, such as adaptive sensory feedback that
responds dynamically to user input, to further push the boundaries of immersive virtual
reality experiences.
77

APPENDIX A
Questionnaires
Questionnaires: User Experience in Virtual Reality
Thank you for participating in this study. Please answer the following questions based
on your experience with the virtual reality system. Your responses will help us evaluate
and improve the VR experience.
Participant Information
1. Please enter your name: _____________________
2. Please enter your age: _____________________
3. Do you have experience with VR?
Options: Yes / No
Presence
4. How strongly did you feel ’present’ in the virtual environment during the experience?
Options: Not at all present / Slightly present / Moderately present / Very present
/ Completely present
5. To what extent did you feel like you were ’inside’ the virtual world rather than
merely observing it?
Options: Not at all / Slightly / Moderately / Very much / Completely
6. How aware were you of your actual physical surroundings while in the VR environ-
ment?
Options: Not aware at all / Slightly aware / Moderately aware / Very aware /
Completely aware
79
A. Questionnaires
Immersion
7. How effectively did the virtual environment engage your senses?
Options: Not effective at all / Slightly effective / Moderately effective / Very
effective / Extremely effective
8. How would you rate your level of immersion while being in the virtual environment?
Options: Not immersive at all / Slightly immersive / Moderately immersive / Very
immersive / Completely immersive
9. How engaging did you find the VR experience overall?
Options: Not engaging at all / Slightly engaging / Moderately engaging / Very
engaging / Extremely engaging
10. To what degree did the virtual environment replace your real-world sensory experi-
ences?
Options: Not at all / Slightly / Moderately / Very much / Completely
11. Did you lose track of time while in the virtual environment? Please describe your
experience.
Open-ended response:
Motion Sickness
12. Did you experience any symptoms of motion sickness during the VR experience?
Options: Yes / No
13. How would you rate the severity of your motion sickness symptoms during the VR
experience?
Options: None / Mild / Moderate / Severe / Extremely severe
14. If yes, please specify the symptoms you experienced.
Options: Nausea / Dizziness / Headache / Disorientation / Other (please specify)
Emotional Response
15. How pleasant or unpleasant did you find the VR experience?
Options: Very unpleasant / Slightly unpleasant / Neutral / Slightly pleasant / Very
pleasant
16. How stimulating did you find the VR experience?
Options: Not stimulating at all / Slightly stimulating / Moderately stimulating /
Very stimulating / Extremely stimulating
17. How did the VR experience make you feel?
Open-ended response:
80
18. Do you have a specific emotion that is connected to the VR experience?
Open-ended response:
19. How intense were your emotional reactions during the VR experience?
Options: Not intense at all / Slightly intense / Moderately intense / Very intense /
Extremely intense
Sensory Feedback (Group B Only)
20. Rate the effectiveness of the tactile feedback (e.g., vibrations) in contributing to
the realism of the VR experience.
Options: Not effective at all / Slightly effective / Moderately effective / Very
effective / Extremely effective
21. How well did the tactile sensations match what you saw and heard in the virtual
environment?
Options: Not at all / Slightly / Moderately / Very well / Perfectly
22. To what extent did the sound increase your engagement or immersion in the virtual
environment?
Options: Not at all / Slightly / Moderately / Very much / Completely
23. Were there any aspects of the sensory feedback that you found confusing or less
intuitive?
Open-ended response:
Feedback and Suggestions
24. What did you like most about the experience?
Open-ended response:
25. What did you like least about the experience?
Open-ended response:
26. Do you have any suggestions for improvement?
Open-ended response:
81

List of Figures
2.1 VR components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 Lanier’s data glove . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 CAVE system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4 HTC and Apple HMDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5 description of binaural audio . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.6 VR setup example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.1 Sketch for physical device . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2 Feedback loop, src: Sebastian Reissmüller . . . . . . . . . . . . . . . . . . 30
3.3 Image of la muralla roja . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.4 Sketch for the virtual world . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.1 Rendering - source: Sebastian Reissmueller . . . . . . . . . . . . . . . . . 40
4.2 Blender - architecture rendering . . . . . . . . . . . . . . . . . . . . . . . . 41
4.3 Ableton sound composition . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.4 Effects in Ableton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.5 Physical component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.6 Physical construction in final form . . . . . . . . . . . . . . . . . . . . . . 47
4.7 Render view of improved system . . . . . . . . . . . . . . . . . . . . . . . 54
4.8 Final simulation run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.1 Trial run of the user test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.2 Presence graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
5.3 Immersion graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.4 Motion sickness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.5 Emotional response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
83

Bibliography
[1] Vreeclimber Contents: Phantastic Sound Experience – Research Unit Vir-
tual Augmented Reality. URL https://www.vr.tuwien.ac.at/topics/
vreeclimber-contents-phantastic-sound-experience/.
[2] The Climb - A stunning VR rock climbing game. URL https://www.crytek.
com/games/climb.
[3] Rolf S. Adelsberger, Alberto Calatroni, and Salar Shahna. A novel piezo-based
technology for haptic feedback for xr. In 2023 IEEE Conference on Virtual Reality
and 3D User Interfaces Abstracts and Workshops (VRW), pages 1015–1016, 2023.
doi: 10.1109/VRW58643.2023.00353.
[4] Eelke Folmer* Aniruddha Prithul, Isayas Berhe. Teleportation in virtual reality; a
mini-review. Front. Virtual Real.,Sec. Technologies for VR, page 18, 10 2021. doi:
https://doi.org/10.3389/frvir.2021.730792.
[5] Christoph Anthes, Rubén García Hernandez, Markus Wiedemann, and Dieter
Kranzlmüller. State of the art of virtual reality technologies. In State of the Art of
Virtual Reality Technologies, 03 2016. doi: 10.1109/AERO.2016.7500674.
[6] Christoph Anthes, Rubén García Hernandez, Markus Wiedemann, and Dieter
Kranzlmüller. State of the art of virtual reality technologies. In State of the Art of
Virtual Reality Technologies, 03 2016. doi: 10.1109/AERO.2016.7500674.
[7] Apple. Apple vision pro. Apple Website, 2024. URL https://www.apple.com/
apple-vision-pro/.
[8] Apple. vision pro. Apple Website, 2024. Available at https://www.apple.com/
apple-vision-pro/, Online; accessed: 20-07-2024.
[9] Ferran Argelaguet and Carlos Andújar. Visual feedback techniques for virtual
pointing on stereoscopic displays. In Proceedings of the ACM Symposium on Virtual
Reality Software and Technology, pages 163–170, 11 2009. doi: 10.1145/1643928.
1643966.
85
[10] Ingo Assenmacher, Torsten Kuhlen, Tobias Lentz, and Michael Vorlaender. In-
tegrating real-time binaural acoustics into vr applications. In 10th Eurographics
Symposium on Virtual Environments, pages 129–136, 06 2004. doi: 10.2312/EGVE/
EGVE04/129-136.
[11] Doug A Bowman, Ryan P McMahan, and Eric D Ragan. Questioning naturalism in
3D user interfaces. Commun. ACM, 55(9):78–88, September 2012.
[12] James Broderick, Jim Duggan, and Sam Redfern. The importance of spatial audio
in modern games and virtual environments. In 2018 IEEE Games, Entertainment,
Media Conference (GEM). IEEE, August 2018.
[13] F.P. Brooks. What’s real about virtual reality? IEEE Computer Graphics and
Applications, 19(6):16–27, 1999. doi: 10.1109/38.799723.
[14] Bruun-Pedersen. Vr setup. Researchgate, 2018. Available at https:
//www.researchgate.net/figure/Schematic-of-the-VR-setup_
fig4_325488813, Online; accessed: 15-06-2024.
[15] Jon Bruun-Pedersen, Morten Andersen, Mathias Clemmensen, Mads Didriksen,
Emil Wittendorff, and Stefania Serafin. The Effect of Multimodal Feedback on
Perceived Exertion on a VR Exercise Setting, pages 12–30. Springer, 06 2018. ISBN
978-3-319-91583-8. doi: 10.1007/978-3-319-91584-5_2.
[16] Maria Castelhano, Diana Almeida, Leonel Morgado, and Daniela Pedrosa. Instruc-
tional design model for virtual reality: Testing and participant experience evaluation.
In Eva Brooks, Anders Kalsgaard Møller, and Emma Edstrand, editors, Design,
Learning, and Innovation, pages 62–75, Cham, 2024. Springer Nature Switzerland.
ISBN 978-3-031-67307-8.
[17] Umer Asghar Chattha, Uzair Iqbal Janjua, Fozia Anwar, Tahir Mustafa Madni,
Muhammad Faisal Cheema, and Sana Iqbal Janjua. Motion sickness in virtual
reality: An empirical evaluation. IEEE Access, 8:130486–130499, 2020. doi: 10.1109/
ACCESS.2020.3007076.
[18] Li-Te Cheng, Rick Kazman, and John Robinson. Vibrotactile feedback in delicate
virtual reality operations. In Proceedings of the Fourth ACM International Conference
on Multimedia, MULTIMEDIA ’96, page 243–251, New York, NY, USA, 1997.
Association for Computing Machinery. ISBN 0897918711. doi: 10.1145/244130.
244220. URL https://doi.org/10.1145/244130.244220.
[19] Gregori Civera. muralla roja. illustrarch, 2019. Available at https://
illustrarch.com/articles/2076-la-muralla-roja.html, Online; ac-
cessed: 22-06-2024.
[20] Natalia Cooper, Ferdinando Milella, Carlo Pinto, Iain Cant, Mark White, and Georg
Meyer. The effects of substitute multisensory feedback on task performance and
86
the sense of presence in a virtual reality environment. PLOS ONE, 13(2):1–25, 02
2018. doi: 10.1371/journal.pone.0191846. URL https://doi.org/10.1371/
journal.pone.0191846.
[21] Carolina Cruz-Neira, Daniel Sandin, T. Defant, Robert Kenyon, and John Hart. The
cave-audio visual experience virtual environment. Communications of The ACM -
CACM, 01 1992.
[22] James Cummings and Jeremy Bailenson. How immersive is enough? a meta-analysis
of the effect of immersive technology on user presence. Media Psychology, 19:1–38,
05 2015. doi: 10.1080/15213269.2015.1015740.
[23] Bram de Smit and Imre Horva´th. Exploring haptic feedback principles for applica-
tion in airborne virtual reality environments. In Volume 3: 28th Computers and
Information in Engineering Conference, Parts A and B. ASMEDC, January 2008.
[24] Heather Desurvire and Max Kreminski. Are game design and user research guidelines
specific to virtual reality effective in creating a more optimal player experience? yes,
vr play. In Aaron Marcus and Wentao Wang, editors, Design, User Experience, and
Usability: Theory and Practice, pages 40–59, Cham, 2018. Springer International
Publishing. ISBN 978-3-319-91797-9.
[25] Julia Diemer, Georg W Alpers, Henrik M Peperkorn, Youssef Shiban, and Andreas
Mühlberger. The impact of perception and presence on emotional reactions: a review
of research in virtual reality. Front. Psychol., 6:26, January 2015.
[26] H.Q. Dinh, N. Walker, L.F. Hodges, Chang Song, and A. Kobayashi. Evaluating the
importance of multi-sensory input on memory and the sense of presence in virtual
environments. In Proceedings IEEE Virtual Reality (Cat. No. 99CB36316), pages
222–228, 1999. doi: 10.1109/VR.1999.756955.
[27] Horst Eidenberger. Smell and touch in the virtual jumpcube. Multimedia Systems,
24, 11 2018. doi: 10.1007/s00530-018-0592-y.
[28] Interaction Design Foundation. What is presence in virtual reality (vr)?. Interaction
Design Foundation - IxDF, 11 2023. URL https://www.interaction-design.
org/literature/topics/presence.
[29] Interaction Design Foundation. What is spatial audio?”. Interaction Design
Foundation - IxDF, 08 2024. URL https://www.interaction-design.org/
literature/topics/spatial-audio.
[30] Eric Foxlin. Motion tracking requirements and technologies. Handbook of Virtual
Environments: Design, Implementation, and Applications, 01 2002.
[31] Greg. binaural audio. binauralhdtracks Website, 2019. Available at https://
binauralhdtracks.com/what-is-binaural-audio/, Online; accessed: 24-
07-2024.
87
[32] Ayah Hamad and Bochen Jia. How virtual reality technology has changed our
lives: An overview of the current and potential applications and limitations. Int. J.
Environ. Res. Public Health, 19(18):11278, September 2022.
[33] Ping-Hsuan Han, Yang-Sheng Chen, Kong-Chang Lee, Hao-Cheng Wang, Chiao-En
Hsieh, Jui-Chun Hsiao, Chien-Hsing Chou, and Yi-Ping Hung. Haptic around. In
Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology,
New York, NY, USA, November 2018. ACM.
[34] Daniel Harley, Alexander Verni, Mackenzie Willis, Ashley Ng, Lucas Bozzo, and Ali
Mazalek. Sensory vr: Smelling, touching, and eating virtual reality. In Proceedings
of the Twelfth International Conference on Tangible, Embedded, and Embodied
Interaction, TEI ’18, page 386–397, New York, NY, USA, 2018. Association for
Computing Machinery. ISBN 9781450355681. doi: 10.1145/3173225.3173241. URL
https://doi.org/10.1145/3173225.3173241.
[35] Scott Hayden. Review: The climb for Oculus
Rift, 10 2018. URL https://www.roadtovr.com/
review-climb-best-looking-vr-game-ive-ever-played-neck-killing/.
[36] Brian Heater. Ten years later, facebook’s oculus acquisition hasn’t changed the
world as expected. Techcrunch, 2024. URL https://techcrunch.com/2024/
04/04/facebooks-oculus-acquisition-turns-10/.
[37] Claudia Hendrix and Woodrow Barfield. The Sense of Presence within Auditory
Virtual Environments. Presence: Teleoperators and Virtual Environments, 5(3):
290–301, 08 1996. doi: 10.1162/pres.1996.5.3.290. URL https://doi.org/10.
1162/pres.1996.5.3.290.
[38] HTC. Viveoverview. VIVE Website, access: 10-07-2024. Available at https:
//www.vive.com/us/product/vive-pro2/overview/, version 1.6.0.
[39] IDF. What is presence in virtual reality (VR) — updated 2024. https://www.
interaction-design.org/literature/topics/presence, nov 2023. On-
line; Accessed: 15-08-2024.
[40] IDF. What is field of view (FoV) in extended reality? — updated
2024. https://www.interaction-design.org/literature/topics/
field-of-view-fov-in-extended-reality, December 2023. Accessed:
2024-9-3.
[41] Kristine Jørgensen Isak de Villiers Bosman, Oğuz ‘Oz’ Buruk and Juho Hamari. The
effect of audio on the experience in virtual reality: a scoping review. Behaviour &
Information Technology, 43(1):165–199, 2024. doi: 10.1080/0144929X.2022.2158371.
URL https://doi.org/10.1080/0144929X.2022.2158371.
88
[42] Mohd Javaid and Abid Haleem. Virtual reality applications toward medical
field. Clinical Epidemiology and Global Health, 8(2):600–605, 2020. ISSN 2213-
3984. doi: https://doi.org/10.1016/j.cegh.2019.12.010. URL https://www.
sciencedirect.com/science/article/pii/S2213398419304294.
[43] Jason Jerald. The VR Book: Human-Centered Design for Virtual Reality. Association
for Computing Machinery and Morgan & Claypool, 2015. ISBN 9781970001129.
[44] Anton Kaplanyan, Anton Sochenov, Thomas Leimkühler, Mikhail Okunev, Todd
Goodall, and Gizem Rufo. Deepfovea: neural reconstruction for foveated rendering
and video compression using learned statistics of natural videos. ACM Transactions
on Graphics, 38:1–13, 11 2019. doi: 10.1145/3355089.3356557.
[45] Angelika C Kern and Wolfgang Ellermeier. Audio in VR: Effects of a soundscape
and movement-triggered step sounds on presence. Front. Robot. AI, 7:20, February
2020.
[46] Sumin Kim, Krzysztof Izdebski, and Peter König. The Effectiveness of Multimodal
Sensory Feedback on VR Users’ Behavior in an L-Collision Problem, pages 381–
389. Springer International Publishing, 01 2019. ISBN 978-3-030-19273-0. doi:
10.1007/978-3-030-18715-6_32.
[47] Kafka JX Van Eickels RL Plener PL Felnhofer A. Kothgassner OD, Goreis A. Virtual
reality exposure therapy for posttraumatic stress disorder (ptsd): a meta-analysis.
Eur J Psychotraumatol., 2019. doi: 10.1080/20008198.2019.1654782.
[48] Jaron Lanier and Frank Biocca. An Insider’s View of the Future of Virtual Reality.
Journal of Communication, 42(4):150–172, 02 2006. ISSN 0021-9916. doi: 10.1111/
j.1460-2466.1992.tb00816.x. URL https://doi.org/10.1111/j.1460-2466.
1992.tb00816.x.
[49] Jeroen S Lemmens, Monika Simon, and Sindy R Sumter. Fear and loathing in VR:
the emotional and physiological effects of immersive games. Virtual Real., 26(1):
223–234, March 2022.
[50] Palmer Luckey. How oculus rift was born. The verge, 2013.
:https://www.theverge.com/2013/1/7/3848914/oculus-rift-deep-inside-the-
immersive-disorienting-virtual-reality; Online; Accessed: 11.09.2024.
[51] Marijn Mado, Géraldine Fauville, Hanseul Jun, Elise Most, Carlyn Strang, and
Jeremy N. Bailenson. Accessibility of Educational Virtual Reality for Children
During the COVID-19 Pandemic. Technology, Mind, and Behavior, 3(1: Spring
2022), mar 15 2022. https://tmb.apaopen.org/pub/g9jmbwyl.
[52] Ryan McMahan, Doug Bowman, David Zielinski, and Rachael Brady. Evaluating
display fidelity and interaction fidelity in a virtual reality game. IEEE transactions
on visualization and computer graphics, 18:626–33, 04 2012. doi: 10.1109/TVCG.
2012.43.
89
[53] Miguel Melo, Guilherme Goncalves, Pedro Monteiro, Hugo Coelho, Jose Vasconcelos-
Raposo, and Maximino Bessa. Do multisensory stimuli benefit the virtual reality
experience? a systematic review. IEEE Trans. Vis. Comput. Graph., 28(2):1428–1442,
February 2022.
[54] Jeitziner MM. Knobel S.E.J. et al. Naef, A.C. Investigating the role of auditory and
visual sensory inputs for inducing relaxation during virtual reality stimulatio. Sci
Rep, 12, 2022. ISSN 17073. doi: https://doi.org/10.1038/s41598-022-21575-9. URL
https://www.nature.com/articles/s41598-022-21575-9#citeas.
[55] D. S. Pamungkas and K. Ward. Electro-Tactile feedback system to enhance virtual
reality experience. International Journal of Computer Theory and Engineering, 8
(6):465–470, 12 2016. doi: 10.7763/ijcte.2016.v8.1090. URL https://doi.org/
10.7763/ijcte.2016.v8.1090.
[56] Kranti Parida, Siddharth Srivastava, and Gaurav Sharma. Beyond mono to binaural:
Generating binaural audio from mono audio with depth and cross modal attention.
In Beyond Mono to Binaural, pages 2151–2160, 01 2022. doi: 10.1109/WACV51458.
2022.00221.
[57] Pausch, Dennis Proffitt, and D. Williams. Quantifying immersion in virtual reality.
Proceedings of SIGGRAPH?97, 13-18, 08 1997. doi: 10.1145/258734.258744.
[58] Marta Pizzolante, Sabrina Bartolotta, Eleonora Diletta Sarcinella, Alice Chirico,
and Andrea Gaggioli. Virtual vs. real: exploring perceptual, cognitive and affective
dimensions in design product experiences. BMC Psychol., 12(1):10, January 2024.
[59] Thomas Potter, Zoran Cvetković, and Enzo De Sena. On the relative importance
of visual and spatial audio rendering on VR immersion. Front. Signal Process., 2,
September 2022.
[60] Vida Rahani, Alireza Vard, and Mostafa Najafi. Claustrophobia game: Design and
development of a new virtual reality game for treatment of claustrophobia. Journal
of Medical Signals and Sensors, 8:231–237, 10 2018. doi: 10.4103/jmss.JMSS_27_18.
[61] Head related transfer function. Head-related transfer function— Wikipedia,
the free encyclopedia, 2024. URL https://en.wikipedia.org/wiki/
Head-related_transfer_function. [Online; Accessed: 10-09-2024].
[62] Simon Riches, Priyanga Jeyarajaguru, Lawson Taylor, Carolina Fialho, Jordan
Little, Lava Ahmed, Aileen O’Brien, Catheleine van Driel, Wim Veling, and Lucia
Valmaggia. Virtual reality relaxation for people with mental health conditions: a
systematic review. Soc. Psychiatry Psychiatr. Epidemiol., 58(7):989–1007, July 2023.
[63] George Robertson, Mary Czerwinski, and Maarten Dantzich. Immersion in desktop
virtual reality. In Proceedings of the 10th annual ACM symposium on User interface
software and technology, pages 11–19, 01 1997. doi: 10.1145/263407.263409.
90
[64] Carl Schissler, Aaron Nicholls, and Ravish Mehra. Efficient hrtf-based spatial audio
for area and volumetric sources. IEEE Transactions on Visualization and Computer
Graphics, 22(4):1356–1366, 2016. doi: 10.1109/TVCG.2016.2518134.
[65] Berenice Serrano, Rosa M. Baños, and Cristina Botella. Virtual reality and stim-
ulation of touch and smell for inducing relaxation: A randomized controlled
trial. Computers in Human Behavior, 55:1–8, 2016. ISSN 0747-5632. doi:
https://doi.org/10.1016/j.chb.2015.08.007. URL https://www.sciencedirect.
com/science/article/pii/S0747563215300856.
[66] Lindsay Alexander Shaw, Burkhard Claus Wuensche, Christof Lutteroth, Jude
Buckley, and Paul Corballis. Evaluating sensory feedback for immersion in exergames.
In Proceedings of the Australasian Computer Science Week Multiconference, ACSW
’17, New York, NY, USA, 2017. Association for Computing Machinery. ISBN
9781450347686. doi: 10.1145/3014812.3014823. URL https://doi.org/10.
1145/3014812.3014823.
[67] Yuxiang Shi and Guozhen Shen. Haptic sensing and feedback techniques toward
virtual reality. Research (Wash. D.C.), 7:0333, March 2024.
[68] Mel Slater and Sylvia Wilbur. A framework for immersive virtual environ-
ments (five): Speculations on the role of presence in virtual environments. Pres-
ence: Teleoperators & Virtual Environments, 6:603–616, 1997. URL https:
//api.semanticscholar.org/CorpusID:9437981.
[69] Ivan E. Sutherland. The ultimate display, 1965. URL https://api.
semanticscholar.org/CorpusID:126382308.
[70] Ivan E. Sutherland. A head-mounted three dimensional display. In Proceedings
of the December 9-11, 1968, Fall Joint Computer Conference, Part I, AFIPS ’68
(Fall, part I), page 757–764, New York, NY, USA, 1968. Association for Computing
Machinery. ISBN 9781450378994. doi: 10.1145/1476589.1476686. URL https:
//doi.org/10.1145/1476589.1476686.
[71] Christoph Tremmel. Vr panel. Researchgate, 2019. Available at
https://www.researchgate.net/publication/337637720_EEG_
Spectral_Conditioning_for_Cognitive-state_Classification_
in_Interactive_Virtual_Reality, Online; accessed: 15-07-2024.
[72] virtuix. Omni by Virtuix | The leading and most popular VR treadmill. URL
https://www.virtuix.com/.
[73] HTC VIVE. vivepro. Vive Website, 2024. Available at https://www.vive.com/
us/product/vive-pro2/overview/, Online; accessed: 20-07-2024.
[74] Vtieto. Case study - My first year on the HoloLens de-
sign team - Mixed Reality, 8 2022. URL https://learn.
91
microsoft.com/en-us/windows/mixed-reality/out-of-scope/
case-study-my-first-year-on-the-hololens-design-team.
[75] Cecilie Våpenstad, Erlend Fagertun Hofstad, Thomas Langø, Ronald Mårvik, and
Magdalena Karolina Chmarra. Perceiving haptic feedback in virtual reality simulators.
Surgical Endoscopy, 27(7):2391–2397, 1 2013. doi: 10.1007/s00464-012-2745-y. URL
https://doi.org/10.1007/s00464-012-2745-y.
[76] Nadine Wagener, Johannes Schoning, Yvonne Rogers, and Jasmin Niess. Letting
it go: Four design concepts to support emotion regulation in virtual reality. In
2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and
Workshops (VRW). IEEE, March 2023.
[77] Nadine Wagener, Johannes Schöning, Yvonne Rogers, and Jasmin Niess. Letting
it go: Four design concepts to support emotion regulation in virtual reality. In
2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and
Workshops (VRW), pages 763–764, 03 2023. doi: 10.1109/VRW58643.2023.00224.
[78] Ingo Wald, Timothy Purcell, J. Schmittler, Carsten Benthin, and Philipp Slusallek.
Realtime ray tracing and its use for interactive global illumination. Eurographics
State of the Art Reports, 01 2003.
[79] Stefan Weber, David Weibel, and Fred W. Mast. How to get there when you
are there already? defining presence in virtual reality and the importance of
perceived realism. Frontiers in Psychology, 12, 2021. ISSN 1664-1078. doi: 10.
3389/fpsyg.2021.628298. URL https://www.frontiersin.org/journals/
psychology/articles/10.3389/fpsyg.2021.628298.
[80] Chyanna Wee, Kian Meng Yap, and Lim Ning. Haptic interfaces for virtual reality:
Challenges and research directions. IEEE Access, PP:1–1, 08 2021. doi: 10.1109/
ACCESS.2021.3103598.
[81] Wikipedia. dataglove. Wikipedia, 2009. Available at https://de.wikipedia.
org/wiki/Datenhandschuh, Online; accessed: 12-07-2024.
[82] Wikipedia. Cave. Wikipedia, 2024. Available at https://de.wikipedia.
org/wiki/Cave_Automatic_Virtual_Environment, Online; accessed: 18-
07-2024.
[83] Wikipedia contributors. Immersion (virtual reality) — Wikipedia, the free en-
cyclopedia, 2024. URL https://en.wikipedia.org/w/index.php?title=
Immersion_(virtual_reality)&oldid=1248502371. [Online; accessed 11-
09-2024].
[84] Wikipedia contributors. Virtual reality sickness — Wikipedia, the free encyclopedia,
2024. URL https://en.wikipedia.org/w/index.php?title=Virtual_
reality_sickness&oldid=1255168203. [Online; accessed 22-08-2024].
92
[85] Wikipedia contributors. Google earth — Wikipedia, the free encyclopedia, 2024.
URL https://en.wikipedia.org/w/index.php?title=Google_Earth&
oldid=1258539990. [Online; accessed 12-08-2024].
[86] Wikipedia contributors. Software engineering — Wikipedia, the free encyclopedia,
2024. URL https://en.wikipedia.org/w/index.php?title=Software_
engineering&oldid=1257359031. [Online; accessed 22-09-2024].
[87] Wikipedia contributors. Virtual reality headset — Wikipedia, the free en-
cyclopedia. https://en.wikipedia.org/w/index.php?title=Virtual_
reality_headset&oldid=1256144462, 2024. [Online; accessed 24-09-2024].
[88] Soranzo A. Wilson, C. J. The use of virtual reality in psychology: A case study
in visual perception. Computational and mathematical methods in medicine, -,
2015. ISSN 151702. doi: https://doi.org/10.1155/2015/151702. URL https:
//www.ncbi.nlm.nih.gov/pmc/articles/PMC4538594/.
[89] A Witmer and Mel Slater. Measuring presence: A response to the witmer and singer
presence questionnaire. Presence (Camb.), 8, 12 1999. doi: 10.1162/105474699566477.
[90] Bob G Witmer and Michael J Singer. Measuring presence in virtual environments:
A presence questionnaire. Presence (Camb.), 7(3):225–240, June 1998.
[91] Shengjie Yao and Gyoung Kim. The effects of immersion in a virtual reality game:
Presence and physical activity. In Lecture Notes in Computer Science, Lecture notes
in computer science, pages 234–242. Springer International Publishing, Cham, 2019.
[92] M. Zyda. From visual simulation to virtual reality to games. Computer, 38(9):25–32,
2005. doi: 10.1109/MC.2005.297.
93