Technische Universität Wien A-1040 Wien ▪ Karlsplatz 13 ▪ Tel. +43-1-58801-0 ▪ www.tuwien.ac.at Development of an Active Motion Capture Suit for Teaching Motion Skills DIPLOMARBEIT zur Erlangung des akademischen Grades Diplom-Ingenieur im Rahmen des Studiums Medieninformatik eingereicht von Emanuel Vonach und Georg Gerstweiler Matrikelnummer 0225307 und 0225028 an der Fakultät für Informatik der Technischen Universität Wien Betreuung Betreuer: Priv.-Doz. Mag. Dr. Hannes Kaufmann, Univ. Ass. Wien, 14.10.2011 (Unterschrift Verfasser) (Unterschrift Betreuer) Die approbierte Originalversion dieser Diplom-/Masterarbeit ist an der Hauptbibliothek der Technischen Universität Wien aufgestellt (http://www.ub.tuwien.ac.at). The approved original version of this diploma or master thesis is available at the main library of the Vienna University of Technology (http://www.ub.tuwien.ac.at/englweb/). i Gerstweiler, Vonach, 2011 Erklärung zur Verfassung der Arbeit Emanuel Vonach 1220 Wien „Hiermit erkläre ich, dass ich die mit meinem Namen markierten Teile der Arbeit selbstständig verfasst habe, dass ich die verwendeten Quellen und Hilfsmittel vollständig angegeben habe und dass ich die Stellen der Arbeit – einschließlich Tabellen, Karten und Abbildungen –, die anderen Werken oder dem Internet im Wortlaut oder dem Sinn nach entnommen sind, auf jeden Fall unter Angabe der Quelle als Entlehnung kenntlich gemacht habe.“ Wien, am 14.10.2011, ___________________________ Gerstweiler, Vonach, 2011 ii Abstract Current multimedia support for teaching and practicing motion skills is usually limited to video and two-dimensional graphics. Especially in areas like rehabilitation and sports the employment of virtual reality applications for educational purposes could offer considerable benefits, like a detailed real-time feedback about the training progress or the support of motivation and autonomy of the users. However for any virtual reality motion skill training system a means for motion input is essential. In particular application areas related to sports impose a number of special requirements that have to be considered. Most full body motion capture devices are either specifically designed for a certain application or not suitable for sports due to different reasons. For example in a lot of sporting activities it might be required to lie on the floor, perform rolls or make direct contact with other tracked persons. For that reason motion capture techniques where the user has to be equipped with relatively large devices could cause pain if making physical contact to hard surfaces like the floor. Furthermore matters like possible fast movements, sufficient freedom of motion and hygienic issues have to be considered as well. The authors show in the course of this work, that a motion capture suit with active infrared optical markers can be constructed to meet the special challenges of motion skill training. For that purpose they compile a wide range of related requirements and devise concepts to fulfill these needs. Subsequently these concepts are applied to construct a fully functional prototype, suitable for a broad range of sporting activities. In order to assess the performance of the active motion suit in an educational context and to demonstrate the potential of the employment of a virtual environment, the constructed input device is used in an actual virtual reality application for teaching motion skills. The active motion capture suit developed in this project is suitable for sports and rehabilitation, but not limited to these specific application areas. Most importantly the authors successfully accomplished to incorporate all required wiring and electronic components unobtrusive. Due to individually controllable markers the layout can be optimized for different setups and tracking algorithms. Keywords: Motion Skill Training, Active Optical Motion Suit, Full Body Motion Capturing, Active Infrared Markers, Virtual Reality iii Gerstweiler, Vonach, 2011 Kurzfassung Die multimediale Unterstützung für das Lehren und Ausüben von motorischen Fähigkeiten ist meist auf Videos oder Grafiken beschränkt. Vor allem in Bereichen wie Rehabilitation oder Sport könnte der Einsatz von Virtual Reality Anwendungen für pädagogische Zwecke beachtliche Vorteile eröffnen, wie z.B. detailliertes Feedback über Trainingsfortschritte in Echtzeit oder das Fördern der Motivation und Selbstständigkeit der Anwender. Aber unverzichtbar für Virtual Reality ist eine Möglichkeit zum Erfassen der Bewegungsdaten. Die meisten Motion Capture Systeme sind entweder auf eine bestimmte Anwendung zugeschnitten oder aus verschiedenen Gründen für den Bereich Sport ungeeignet. Gerade bei sportlichen Anwendungen treten spezielle Anforderungen auf. Zum Beispiel kann es bei Kampfsportarten nötig sein eine Rolle auszuführen oder in direkten Körperkontakt mit anderen Personen zu treten, deren Position ebenfalls aufgezeichnet wird. Methoden bei denen der Anwender mit verhältnismäßig großen Vorrichtungen ausgestattet ist, könnten dabei zu Schmerzen bei physischem Kontakt mit harten Oberflächen führen. Des Weiteren müssen auch mögliche schnelle Bewegungen, ausreichende Bewegungsfreiheit und hygienische Aspekte bedacht werden. Die Autoren werden im Rahmen dieser Arbeit zeigen, dass ein Anzug für Motion Capture mit aktiven infraroten Markern gebaut werden kann, der den speziellen Herausforderungen des Trainings motorischer Fähigkeiten gerecht wird. Zu diesem Zweck werden Anforderungen zusammengestellt und Konzepte entwickelt um diese zu erfüllen. Anschließend werden diese angewendet um einen voll funktionsfähigen Prototyp zu bauen, welcher für ein weites Spektrum sportlicher Aktivitäten geeignet ist. Um die Leistung des aktiven Motion Capture Anzugs in einem pädagogischen Kontext einschätzen zu können und das Potential des Einsatzes von Virtual Reality zu demonstrieren, wurde der konstruierte Anzug in einer tatsächlichen Virtual Reality Anwendung für das Lehren von motorischen Fähigkeiten verwendet. Es zeigt sich, dass der in diesem Projekt entwickelte Motion Capture Anzug, für Sport und Rehabilitation geeignet ist, sich aber nicht auf diese spezifischen Anwendungsbereiche beschränkt. Den Autoren ist es vor allem gelungen die nötige Verkabelung und die elektronischen Komponenten unscheinbar zu integrieren. Außerdem kann das Layout aufgrund der individuell steuerbaren Marker für unterschiedliche Systeme und Algorithmen optimiert werden. Schlüsselwörter: Training motorischer Fähigkeiten, Rehabilitation und Sport, aktiver optischer Motion Capture Anzug, Ganzkörper Motion Capture, aktive infrarote Marker, Virtual Reality Gerstweiler, Vonach, 2011 iv Acknowledgements There are a lot of people who helped to make this work possible and whom I want to express my sincere gratitude at this point: First of all I want to thank Hannes for his excellent supervision of this thesis and for his mentoring and unlimited patience during this time. Further thanks go to Christian, Mathis, Michael and Thomas for their continuous support and especially for their assistance with the tracking system and in electronic matters. I also want to thank the “Maxim Gesellschaft Für Elektrische Integrierte Bausteine GmbH” in Germany, who were very helpful with answering questions and provided several samples of potentially suitable LED display drivers for testing free of charge. Of course I owe all the participants of the user tests a debt of gratitude for their readiness to volunteer, since a very important part of this thesis depended on it. Heartfelt thanks go to my partner Katharina for her infinite sympathy and love. Her unconditional support and her ever motivating words were essential to this work. Furthermore I want to thank my parents, my family and my friends, who all contributed in various ways to making this thesis possible respectively to my dedication to my studies. And last but not least I thank Georg and Gergely aka Schorsch, whom I both met at my very first day at the university as fellow students and who grew to be my trusted friends throughout my studies and beyond. Since this thesis is a joint work with Georg, I want to thank him in particular for his productive cooperation and his power of endurance. Although our ideas are often contrary, they somehow always merge through endless discussions to remarkable concepts. v Gerstweiler, Vonach, 2011 Table of Contents 1. Introduction .................................................................................................. 1  1.1. Introduction to Virtual Reality ........................................................................... 2  2. Main Objectives ............................................................................................ 6  2.1. Central Hypothesis ............................................................................................ 6  2.2. Research Objectives .......................................................................................... 7  2.3. Limitations .......................................................................................................... 7  3. Related Work ................................................................................................ 9  3.1. Motion Tracking Technologies ......................................................................... 9  3.1.1. Mechanical Tracking ................................................................................... 10 3.1.2. Magnetic Tracking ....................................................................................... 11 3.1.3. Acoustic Tracking ........................................................................................ 13 3.1.4. Inertial Tracking ........................................................................................... 14 3.1.5. Optical Tracking .......................................................................................... 16 3.1.6. Hybrid Tracking ........................................................................................... 18 3.2. Concepts Related to Education and Rehabilitation Applications ............... 19  3.3. Analysis and Motivation .................................................................................. 21  4. Discussion of Requirements and Technologies ..................................... 23  4.1. Requirements of a tracking system for teaching motion skills ................... 23  4.2. Tracking Technology Decision ....................................................................... 26  4.3. VR-Setup ........................................................................................................... 30  4.4. Skeleton Estimation Algorithm ....................................................................... 31  4.5. Marker Layout ................................................................................................... 33  5. Electronic Fundamentals and Communication Protocols ...................... 38  5.1. Light Emitting Diode ........................................................................................ 38  5.2. LED Matrix Circuit ............................................................................................ 40  5.3. Galvanic Coupling ............................................................................................ 41  5.4. Serial Peripheral Interface – SPI ..................................................................... 42  Gerstweiler, Vonach, 2011 vi 5.5. Inter-Integrated Circuit – I²C ........................................................................... 44  6. Motion Suit - Conceptual Design .............................................................. 47  6.1. Material ............................................................................................................. 48  6.1.1. Base Material .............................................................................................. 48 6.1.2. Conductive Thread ..................................................................................... 50 6.1.3. Insulation .................................................................................................... 52 6.1.4. Connectors ................................................................................................. 54 6.1.5. IR-Emitter ................................................................................................... 55 6.1.6. Emission Regulation ................................................................................... 58 6.2. Control Box Components ............................................................................... 59  6.2.1. LED Matrix Driver ....................................................................................... 60 6.2.2. Controlling Unit ........................................................................................... 63 6.2.3. Power Source and Consumption ................................................................ 65 6.2.4. Additional Components .............................................................................. 66 6.3. Software Architecture for Operation and Management ............................... 67  7. Motion Suit – Construction ........................................................................ 71  7.1. Feasibility Tests .............................................................................................. 71  7.2. Suit Adaptations .............................................................................................. 77  7.3. Wiring ............................................................................................................... 81  7.4. Connectors and Connections ........................................................................ 85  7.5. Active Markers ................................................................................................. 89  7.6. Control Box ...................................................................................................... 91  7.7. Software for Operation and Management ..................................................... 95  7.7.1. Firmware ..................................................................................................... 96 7.7.2. Motion Suit Manager .................................................................................. 99 7.8. Costs .............................................................................................................. 101  10. Evaluation of the Motion Suit ................................................................ 103  10.1. Setup of User Tests ..................................................................................... 103  10.2. User Test Observations .............................................................................. 109  10.3. Results ......................................................................................................... 111  vii Gerstweiler, Vonach, 2011 11. Conclusion and Future Work ................................................................ 113  Appendix ....................................................................................................... 115  A. Electrical Fundamentals ......................................................................... 115  A.1. Voltage (V) ...................................................................................................... 115  A.2. Current (A) ...................................................................................................... 116  A.3. Resistance (R) ................................................................................................ 117  A.4. Electric Power (P) .......................................................................................... 118  A.5. Ohm’s Law ..................................................................................................... 118  A.6. Capacitor ........................................................................................................ 120  A.7. Series and Parallel Circuits .......................................................................... 121  A.7.1. Series Circuit ............................................................................................ 121 A.7.2. Parallel Circuit ........................................................................................... 122 A.7.3. Series-Parallel Circuit ............................................................................... 123 B. Motion Suit Development Software ....................................................... 124  B.1. Processing IDE .............................................................................................. 124  B.2. Arduino IDE .................................................................................................... 125  B.3. EAGLE Light Edition ..................................................................................... 125  C. Chapter - Author Correlation .................................................................. 128  Bibliography ................................................................................................. 135  Introduction Gerstweiler, Vonach, 2011 1 1. Introduction Emanuel Vonach Not long ago “motion capture” and “virtual reality” seemed for most people like terms associated with science fiction. Although the topics are not really new, currently they are rapidly getting part of the everyday life. Especially the game- and movie industry, where the application of motion capture technologies is on the way to become quasi standard, is responsible for bringing the results right into the living room. Yet the spectrum of application areas is not limited to the field of entertainment at all, but on the contrary broadening fast. The work at hand focuses on motion capturing for the application in virtual reality for educational purposes, especially concerning sports and rehabilitation. Current multimedia support for teaching and practicing motion skills is often limited to video or images. The employment of virtual reality applications for educational purposes could open up a whole range of new possibilities, like detailed feedback about the training progress in real time. Furthermore the users might experience increased autonomy and motivation, which both are crucial factors for successful learning [Goudas '95]. However to allow the use of virtual reality for motion skill training, a motion input system is required. Especially in application areas like sports or rehabilitation, there are a number of special requirements and situations that have to be considered. For example when practicing martial arts the user might want to perform a roll, which would lead to pain if he had to be fitted with relatively large devices for motion capturing. In order to demonstrate the benefits of the employment of virtual reality in the context of motion skill training, the authors set themselves the target to construct an active motion capture suit particularly designed for the special challenges of this field and compensating weaknesses of available solutions. For that purpose first a general introduction to the topic is conducted, followed by a clear definition of the research objectives of the project. After that the current state of the art is determined with a thorough literature search. In the course of this, all motion tracking technologies suitable for full body motion capturing are presented including their pros and cons. Subsequently the authors focus on related works specifically with the context of virtual reality for Introduction 2 Gerstweiler, Vonach, 2011 education in sports and rehabilitation. Through the investigation of the already available technologies and concepts it is then possible to infer opportunities for improvements. In this context the authors also present their fundamental motivation for this work. An extensive discussion of the requirements related to the area of motion skill training leads to the decision for the actually applied tracking technology and further to the presentation of the used setup. In order to allow readers who are not especially familiar with this area of expertise to follow the explanations of the authors, an introduction to the employed electronic concepts and protocols is provided. After that the concepts devised by the authors in the course of the project are discussed in detail. Therefore a diversity of components and materials has to be considered and compared. In order to be able to benefit from all the advantages of an active motion capture suit it is further necessary to provide a logical control circuitry as well as a basic software architecture. As a first stage of the actual construction phase, the applicability of the devised concepts is confirmed through a series of feasibility tests conducted by the authors. In the subsequent chapters the focus lies on the implementation of the concepts and the combination of the individual components during the construction of the motion suit. After completion, finally the applicability of the suit for teaching and practicing motion skills is observed during a series of user tests in order to be able to assess the outcome of the project. 1.1. Introduction to Virtual Reality Georg Gerstweiler The concept of virtual reality, in short VR, is developing since at least 1962, when Morton Heilig presented his design for a machine called “Sensorama Simulator” which simulates a motorcycle ride through the city of New York. Almost 50 years ago this device already offered the possibility of displaying 3D video feedback, playing stereo sound, releasing aromas and producing wind effects while the user was sitting on a vibrating seat. But since then both the technology as well as the application requirements have changed. Burdea and Coiffet [Burdea '03] defined the idea of virtual reality as: Introduction Gerstweiler, Vonach, 2011 3 … a high-end user-computer interface that involves real-time simulation and interactions though multiple sensorial channels… [Burdea '03] In contrast to earlier systems a virtual reality user interface gives the possibility to display complex data in a more natural way and therefore takes the human computer interaction to a completely new level [Ellis '89]. The main keywords for VR are “interaction” and “immersion”. These properties can be achieved by keeping the following points in mind [Burdea '03]. When developing a three dimensional simulation the virtual environment should look and behave like the real world, because the user is expecting similar behavior like he is used to in real life while interacting in a simulation application. Real time rendering therefore is an essential technology to give the user a feeling of realism. In order to react to the input from the user, it is necessary to work with a tracking system that is able to determine the exact position and orientation of an object in respect to the user. Any action intended to interact with the environment must have an instantaneous reaction. This does not only involve the visual display, but also all other outputs that can be perceived by a person, like sound, touch, smell or taste. No matter which technology is used to present the data, it has to fit to the resolution of human senses. The visualization of content for example has to guarantee a high frame rate to simulate fluid movements. There are several different virtual reality systems and components, some of them are described in later chapters. In general almost all of these concepts fit to a skeletal architecture designed by Burdea and Coiffet [Burdea '03] (see Figure 1). According to Burdea and Coiffet a classic VR system needs five components which consist of both software and hardware. To enable an interaction between the user and VR system, primarily input devices like a standard mouse or a well designed motion suit are needed to submit instructions to the VR logical part. Output devices on the other hand are required to give useful feedback to the user, like displaying stereoscopic objects. These two parts the input and output devices are forming a basic structure in order to provide a complete communication interface. Introduction 4 Gerstweiler, Vonach, 2011 Figure 1: VR System Components The data acquired by the input devices is then forwarded to another important part of the VR system architecture known as the VR engine, which represents the logical unit. It is working behind the I/O devices and is primarily responsible for the workflow that is needed before the graphical data can be displayed. At this stage all operations have to be executed between two updates of the visual output. It must be ensured that the time in between is still short enough to allow a high frame rate. Before displaying the data, the VR engine needs several steps to analyze the input data of the user. For example if it is necessary to track an object, the VR engine needs a localization and triangulation step before the actual input can be interpreted. At this point, if desired, it is possible to apply rules according to the input and manipulate the virtual environment. Finally the rendering pipeline is able to prepare the visual output. The VR engine not only needs to communicate with the I/O devices but also with the systems “Software and Database” component. This section of the VR System deals with 3D objects that help to simulate a virtual environment. Each object however is more than just a shape, because in order to simulate an environment it is also necessary to define parameters like appearance, kinematic constraints, intelligent behavior and physical characteristics. This design step has a direct influence on the usability of the simulated world. A very detailed simulation results in computational complexity and therefore needs a lot of effort to render the environment in a reasonable time. In order to reduce the complexity and to adapt the visual output to a manageable size, model Introduction Gerstweiler, Vonach, 2011 5 management algorithm can be used to achieve an adequate frame rate and low latencies. When designing a VR application every output respectively feedback the system makes has to be adjusted to the capabilities of human senses [Ellis '89]. The main aim of a virtual reality environment should be to take the immersion of the user to a level so he is able to act like in reality. To achieve this, the user’s brain should be able to accept the virtual environment, which includes a proper feedback adapted to the human senses, as well as a suitable user interaction. It must not be forgotten that this whole procedure is a pure mental process that only occurs in the imagination of the user [Ellis '89]. The last item that is left in this architecture is „Tasks“ which relates to the application areas of VR Systems. They are used in all kind of areas. Special applications which are related to the topic of this thesis will be described in chapter 3.2. The work at hand concentrates mainly on three of the previously mentioned components, namely „I/O Devices“, „Users“ and „Tasks“ (see Figure 1 on the right). First of all there will be special tasks defined by the authors regarding the application area and the users that will be the target group. The subsequently developed device will then be used as an input device of a tracking system. Main Objectives 6 Gerstweiler, Vonach, 2011 2. Main Objectives Emanuel Vonach In the following chapter the main objectives of this project are presented. First the authors formulate and explain the central hypothesis, which they are going to verify in the course of this work. After that they make a clear determination of what they plan to include in their investigations on the one hand, and what will not be considered a goal of this project on the other hand. 2.1. Central Hypothesis Emanuel Vonach The authors formed the following central hypothesis: An active motion suit can fulfill the special requirements that arise in the context of teaching and training arbitrary motion skills. The considerations that led to this statement will be presented in the course of this work: First the current state of the art regarding VR and motion tracking technologies for educational usage in sports or rehabilitation is reviewed. In this context the authors will point out some shortcomings of these approaches and explain their motivation for doing further research in this field. Furthermore in order to investigate this hypothesis, the authors are going to compile the wide range of requirements related to the application area of motion skill training. They will devise concepts to meet those demands and apply them to construct a fully functional prototype of an active motion suit. Observing the performance of the constructed suit during user tests in an educational context should enable the authors to finally corroborate or dismiss the central hypothesis. Main Objectives Gerstweiler, Vonach, 2011 7 2.2. Research Objectives Emanuel Vonach In addition to investigating the central hypothesis, there are a number of further research questions the authors want to explore in this project. First of all not only active optical tracking as possible solution for the already mentioned problems related to motion capturing in sports should be taken into consideration, but also to what extent other available technologies would be suitable. In this context the authors will look into the assets and drawbacks of different approaches and assess their potential for application in the field of interest. Furthermore it is a goal of this project to show that the employment of VR for education in sports and rehabilitation is reasonable and can offer considerable advantages compared to a classic approach, where for example a teacher demonstrates certain complex movements and a student tries to imitate them. For that purpose the authors intend to use the constructed motion suit in an actual VR environment for teaching motion skills and thereby try to prove that this concept can be realized in practice. User tests will be performed to provide feedback in that matter and to find potential improvement opportunities. In contrast to the already available motion capture solutions, the authors consider it important that the suit created in this project should not be limited to a specific use case. In fact it should be suitable for a preferably wide range of sporting and rehabilitation activities, and ideally for other fields besides motion skill training as well. Finally also the fact that the motion capture suit should be usable with a tracking system available at the institute where the research is done might have an influence on individual design steps. 2.3. Limitations Emanuel Vonach In order to be able to focus on the objectives the authors consider most important, clear limitations have to be defined. First of all no applicatory comparison tests between basic motion tracking technologies are to be conducted. The most suitable one for the task of VR motion skill training or Main Objectives 8 Gerstweiler, Vonach, 2011 rehabilitation will be determined through literature search. Furthermore the constructed product will provide all necessary features to examine the assets and drawbacks of the employment of an active motion suit for this area and to prove the previously presented hypothesis, but it is not planned to create a “ready for sale” device. The actual implementation of the mentioned VR application for motion skill training is not part of this work either. The software was developed by the authors in the course of another thesis parallel to this one. In the project at hand it is only used for exemplary demonstration of the pros and cons respectively the suitability of the devised motion suit in the context of motion skill training and rehabilitation. Although it would be very interesting to assess the learning success achieved with VR in contrast to classic methods, this would go far beyond the scope of this work. Extensive studies over a longer period of time would be required to provide reasonable results concerning this matter. Anyway this would have more significance on the employed application than the motion suit or the basic principle. For the same matter extensive user studies are planned neither, but rather profound individual qualitative interviews. They are intended to provide feedback on the applicability of the suit and additionally help to reveal potential opportunities for improvements, not to demonstrate the superiority of one approach above another. Furthermore even though usability will be a criterion while designing the motion suit, it will not be a key aspect. It should be focused on testing basic concepts, not excessive user centered design. Related Work Gerstweiler, Vonach, 2011 9 3. Related Work Emanuel Vonach After the general introduction to the topic and the definition of the objectives of this work, in the following the current state of the art is presented. In particular the assets and drawbacks of the different available tracking technologies suitable for full body motion capturing are examined. After that published concepts regarding motion capturing, especially those with a relation to education in motion skills training and rehabilitation, are briefly reviewed in order to assess the potential contributions and benefits of this work. 3.1. Motion Tracking Technologies Emanuel Vonach In the following the authors give an overview of different available tracking technologies, which are potentially suitable for motion capturing. The information presented in this chapter and its subchapters is based on the books “Virtual Reality Technology” by Burdea and Coiffet [Burdea '03] and “3d User Interfaces: Theory and Practice” by D. A. Bowman [Bowman '05], if not stated otherwise. In VR-Systems information about the three-dimensional location of the user or a physical object is usually required for purposes like view control, motion parallax or object manipulation. Measuring the changes in position and/or orientation of an object without user interaction is called tracking. Common applications are for example motion tracking or eye tracking. In this work we focus on motion capturing, which aims on recording and analyzing movements and creating a correlation between physical and simulated environments. The user’s body or just parts of it are usually mapped to a virtual counterpart, for example an avatar, to allow interaction with the virtual world. The required data, for example position and/or orientation, is sampled preferably in real time by the tracking system and is then used as input to update the virtual scene. The technologies which can be used to achieve this Related Work 10 Gerstweiler, Vonach, 2011 data are widespread and it is not easy to choose the appropriate one for a given task. Therefore tracker performance parameters like the following are used to assess their suitability [Bowman '05, Burdea '03]:  Accuracy: the difference between the tracked position and the actual position in the real world.  Drift: an error that is steadily accumulated over time.  Jitter: an instability in tracker output even if the tracked object is not in motion.  Latency: the time delay between the occurrence of an event (for example a change in position) and the particular moment when it is reported. Total latency includes the time to measure the change, the communication time between tracker and host computer and the time to render and display the updated scene. Latency is also dependent on the update rate.  Update rate: the number of samples that can be taken per second.  Range: the work envelope in which a user can be tracked. Based on [Burdea '03] and [Bowman '05] a variety of tracking technologies with their advantages and drawbacks are presented in the following chapters. 3.1.1. Mechanical Tracking Emanuel Vonach Mechanical trackers were the first to be used in VR Systems. A rigid structure of interconnected linkages with electromechanical sensors, like potentiometers or shaft encoders, is attached to the user or the tracked object, as for example in the “Gypsy 7” by Animazoo [Animazoo '11] (Figure 2). The dimensions of the individual elements of this linkage are known and for absolute tracking one end of the system is fixed at some location in the real world, like the floor or a desk. Based on this information and the measurements of the sensors, position and orientation of the object can be calculated by a host computer. Mechanical trackers are simple to use and potentially very accurate. They are immune to most of the interferences other tracking technologies often suffer Related Work Gerstweiler, Vonach, 2011 11 from, like magnetic fields, reflections or occlusions. In addition they have the lowest latency of all tracking types and jitter is also very low. On the other hand there are some serious disadvantages. Mechanical trackers are generally bulky and limit the user’s freedom of motion. Since at least one end of the system has to be mechanically connected to a fixed location for absolute tracking, the tracking volume is limited to the length of this tracker arm. Therefore to enlarge the tracking range, a longer connection is needed, which results in increased weight and inertia. In addition the tracker arm and the cables can get in the way, reducing the user’s mobility even further. If there is no absolute tracking needed, and the connection to a fixed location can therefore be omitted, mechanical trackers are still ergonomically not ideal. The whole structure has to be supported by the user and the weight of the tracker leads to fatigue, which could diminish the sense of immersion in VR Systems. Figure 2: Animazoo Gypsy 7, a mainly mechanical motion capture system (courtesy of Animazoo). 3.1.2. Magnetic Tracking Emanuel Vonach This tracking technology works with a stationary transmitting device that emits a low frequency magnetic field and at least one receiver which is attached to the Related Work 12 Gerstweiler, Vonach, 2011 tracked object. The magnetic field induces a current in the receiver element, which is sampled and then used to determine the position and orientation in relation to the magnetic source. There are AC and DC magnetic trackers. AC trackers have the unpleasant side effect, that they induce eddy currents in metal objects in the vicinity. As a consequence these objects emit their own magnetic field and distort the one used for tracking. This effect degrades the accuracy. DC magnetic trackers circumvent this problem by inserting a short time delay between the emission of the magnetic field and the sampling of the receiver. This allows most of the eddy currents to dissipate in time. The receivers of magnetic tracking systems are potentially lightweight and small (below 2 cm) and wireless versions are available as well. Unlike some other tracking technologies, no direct line-of-sight between transmitter and sensor is necessary. The range of commercial products usually varies from less than 1 up to 9 m and above, with a tradeoff between range and accuracy. In general smaller receivers allow better accuracy, but less range. For example: the wireless version of “MotionStar” by the Ascension Technology Corporation [Ascension '04] works with up to 80 cubical receivers of a lateral length of 2.54 cm, achieving update rates of 100 Hz (Figure 3). While this system offers an operation range of 3.05 m with a single transmitter, the positional accuracy varies from 0.76 to 1.5 cm and the accuracy of the orientation from 0.5 to 1.0 degrees, in each case depending on the range. The operation radius of magnetic trackers can usually be extended with additional transmitters, but generally accuracy degrades rapidly with the distance. Nevertheless the main disadvantage of magnetic trackers is the high sensitivity to metal. Any ferromagnetic or conductive object present in the room severely distorts the magnetic field. Although DC magnetic trackers can handle the presence of non-ferromagnetic metal like stainless steel or aluminum very well, they are still heavily influenced by objects with high magnetic permeability like copper. The reason for this is that the eddy currents can exist there longer than the time delay used in DC trackers. Calibration and filtering techniques can Related Work Gerstweiler, Vonach, 2011 13 somewhat counteract these problems, at the expense of start-up time and computational overhead. Figure 3: “MotionStar” by Ascension Technology Corporation (courtesy of Ascension). 3.1.3. Acoustic Tracking Emanuel Vonach The speed of sound is known for a certain temperature and can therefore be used to measure distances based on time-of-flight duration. For acoustic tracking a minimum of three receivers (microphones), transmitters (high- frequency, usually ultrasonic, speakers) or both, are used for the triangulation of position and orientation of a point in question (Figure 4a). In case of an “inside- out” approach the microphones are mounted on the tracked object and the emitters are placed stationary in the environment, while the “outside-in” approach works the other way around. Acoustic tracking systems are usually relatively inexpensive, lightweight and simple to use. However there are a lot of disadvantages. The range of such trackers is often quite limited and heavily dependent on air humidity and the Related Work 14 Gerstweiler, Vonach, 2011 desired accuracy. For example the Logitech “Head Tracker” [Logitech '92] has a working area of about 1.5 m in a 100° cone from one transmitter (Figure 4b). It is usually possible to use additional transmitters to enlarge the tracking volume, although this leads to increased complexity as the mutual interferences have to be managed by the host computer. Acoustic trackers permit relatively low sampling rates (about 50 Hz), because there has to be enough time between the sonic pulses to allow any echoes to die. If several transmitters are used, for example for triangulation, or more than one object or body part should be tracked at once, time multiplexing has to be used. This leads to even lower update rates and increased latency. The accuracy of acoustic tracking suffers from reflective surfaces in the vicinity and changes in temperature. Background noise, external sounds like phone ringing or other ultrasonic sources can reduce accuracy significantly too. Problems with jitter, distortions and loss of signal can arise as well. Additionally a direct line-of-sight between transmitter and receiver is required. 3.1.4. Inertial Tracking Emanuel Vonach Inertial trackers work with devices like angular-rate gyroscopes and linear accelerometers to measure the change in orientation and translation velocity of a tracked object. At least three gyroscopes are needed to determine yaw, pitch a. b. Figure 4: a. Logitech “Head Tracker”, b. Tracking Volume (both [Logitech '92]) Related Work Gerstweiler, Vonach, 2011 15 and roll angular velocities, as well as three accelerometers for measuring changes in velocity for each axis of the coordinate system. Based on the starting point, the outputs of these devices allow the calculation of the relative position of the tracked object. This kind of tracking technology offers some considerable advantages. Very high sampling rates are achievable and the range is theoretically unlimited, since wireless versions are available. There are no problems like reflections, external interferences or line-of-sight constraints as with several other technologies and jitter is also very low. An example of a motion capture system using inertial sensing is the “MVN” by Xsens [Xsens '10] (see Figure 5). Figure 5: Xsens “MVN” is an inertial motion capture solution (courtesy of Xsens). Unfortunately there are also a number of downsides. Since inertial tracking is relative to a certain starting point as a matter of principle, for absolute tracking the position of this point has to be obtained with a supplementary technology first. Additionally inertial sensor units with acceptable accuracy are usually not too small, at least a few centimeters in diameter. The actual position of the tracked object is obtained through integration over time. For this reason the bias of the gyroscope causes an error that grows proportionally with time. The error generated by the accelerometer bias increases even with the square of time. Related Work 16 Gerstweiler, Vonach, 2011 This leads to a severe drift problem. To somewhat counteract this issue it is possible to use only orientation, since the drift is less serious with gyroscopes and can therefore be better compensated. Another alternative would be to use a different type of tracker to reset the starting position periodically. Hybrid trackers like this are described in chapter 3.1.6. 3.1.5. Optical Tracking Emanuel Vonach Optical tracking takes advantage of various techniques from computer vision to detect an object with the help of optical features and acquire its position through triangulation of the gathered information. Orientation data is then derived from a set of obtained coordinates. Depending on the particular system a variety of devices and sensors are used. They range from simple cameras over infrared emitters to lateral effect diodes, which are able to detect the direction of incoming light. Additionally it is also possible to create a depth map of the environment to simplify the identification of the objects of interest. This can be achieved by using a stereo pair of cameras or by directly measuring the time-of-flight it takes emitted light to travel between the sensor and a reference point. Another method to acquire such a depth map is applied for Microsoft’s “Kinect” [PrimeSense '11]. It works with a coded infrared pattern that is projected into the tracking volume. When an image of the scene is captured with an optical sensor, the depth information can be retrieved from the coding. The traditional approach for optical tracking is based on an outside-in topology, where optical sensors, typically cameras, are mounted on fixed locations in the environment. The objects or body parts to be tracked are often marked with landmarks, as for example in case of the “iotracker” system [Pintaric '07] (Figure 6). They either actively emit light or passively reflect it from fixed emitters. This enhances detection and generally permits higher accuracy. Quantity and size of these markers vary depending on the particular system. If they are omitted at all, local features like edges or textures of the tracked object are used as reference points. In case of an inside-out approach, the optical sensors are attached to the tracked object. If landmarks such as LED beacons or cards with recognizable Related Work Gerstweiler, Vonach, 2011 17 patterns are to be used, they are placed all over the environment. Advantages of this approach are a higher sensitivity for changes in orientation and the theoretically infinite scalability. Optical tracking permits high update rates and small latencies. Additionally relatively large tracking volumes are possible. Passive or wireless active systems are available which allow the user to move completely unbounded by cables or alike. However in practice numerous optical tracking solutions are wired anyway. Furthermore there are frequently input or output devices in use, which still require a wired connection to the host computer. The range is also limited to some degree by the fact that tracking accuracy degrades with the distance between the markers and the optical sensors. Besides the setup of optical tracking systems can be quite complex, because a number of parameters have to be considered, such as quantity and location of cameras, visual background and possible interfering surfaces, as well as design and placement of the landmarks. Regardless of these challenges, the major drawback of optical tracking systems is the required direct line-of-sight. Especially in motion capture applications one tracked part of the user’s body often occludes another one, which results in a loss of the signal. Additional cameras and markers can minimize this problem, but lead inevitably to an increased complexity. Figure 6: Schematic of the Iotracker System Related Work 18 Gerstweiler, Vonach, 2011 3.1.6. Hybrid Tracking Emanuel Vonach Since every tracking technology has its advantages as well as its flaws, the overall performance of a tracker can be improved significantly through the combination of more than one approach. Thus one method can compensate weaknesses of the other and vice versa. For example the drift of a gyroscope in an inertial tracking system can be compensated with data of the local magnetic north measured by a solid-state magnetometer. Another reasonable match would be to use orientation information measured by inertial sensors combined with position data acquired with acoustic tracking. Even other technologies like GPS can be incorporated, for example outdoors if it is impossible to place emitters or receivers in the environment. As an example Figure 7 shows a combination of mechanical and optical tracking used for the production of the TV series “Sid the Science Kid” [Henson '11]. Hybrid tracking systems can have profound advantages compared to standard methods, at the cost of considerably increased complexity and the downsides that come with it. Figure 7: A hybrid suit used for the television series “Sid the Science Kid” (courtesy of The Jim Henson Company). Related Work Gerstweiler, Vonach, 2011 19 3.2. Concepts Related to Education and Rehabilitation Applications Georg Gerstweiler The last chapter introduced available tracking technologies that are suitable for capturing human motion. Since the main focus in this work addresses an educational usage, it is important to get an overview of the current state of the art in this field, especially regarding VR kinematic education and training systems. A related application area can be found in systems for capturing human motion where the data is applied to a virtual avatar for getting lifelike virtual animations. Such setups usually take advantage of optical tracking systems with ball shaped passive markers as used for example for animating characters in movies or in several video games. For instance the company Electronic Arts [Jeff '09] employed such a system in order to create realistic avatars for a basketball video game. These examples only demonstrate one part which is also required for an educational training system, because in these applications no feedback was needed. But not only in commercial applications such technologies are used: C. Schoenauer et al. [Schönauer '11] for example utilize a passive motion suit for developing a full body motion capturing system, which also considers bio-signals of the user. The outcome of this work is intended to be used for rehabilitation patients. A game engine places the patients, who are wearing the suit, on a virtual island, on which different game levels have to be completed, thereby performing special pre-defined exercises depending on the medical condition. This paper also combines the aspect of motion capturing with motion skills. In the paper “Virtual reality applied to sports” B. Bidau et al. [Bideau '04] (see Figure 8) evaluates the realism of animations by analyzing the reaction of a handball goalkeeper, who faces a synthetic opponent in a virtual environment. During their work they discovered that even small changes to a visualized realistic animation have an effect on the user’s reactions. So realism is an important element when dealing with teaching motion skills. Related Work 20 Gerstweiler, Vonach, 2011 Another work closely related to a virtual training system was published by D. Fitzgerald et al [Fitzgerald '07]. The aim of this project was to develop a cheap motion capturing suit and a feedback system for the area of rehabilitation and sports. In the context of this work they constructed a motion capturing suit with ten inertial sensors (see Figure 9). The educational aspect was realized with the help of a computer game, which instructs the user via audio and visual outputs to perform a specific exercise. The visual output mainly consists out of a virtual avatar, which is animated with a prerecorded motion capturing session. This 3D character encourages the user to mimic the shown sports rehabilitations exercises. After one training session the user’s movements are analyzed. So the user gets an evaluation after the workout has been finished. A similar project was performed by O. Mirabella et al. [Mirabella '11] by measuring movements of a human arm. Again inertial sensors were used to collect the data for animating a virtual arm. This system was also intended to be used for training in sports and rehabilitation, but so far just for a quite small region of interest. Figure 8: A VR application applied in sports using passive markers [Bideau '04]. Figure 9: MoCap Suit with Inertial Sensors [Fitzgerald '07]. D. Miaw [Miaw '10] worked on an approach related to this project at approximately the same time as the authors of this work. He constructed a shirt for motion capturing equipped with several photodetectors. The aim of his thesis was to support motor-learning with the help of a real-time tactile feedback system. Regarding the construction part of his work, the method of using a Related Work Gerstweiler, Vonach, 2011 21 sewed conductive wire for control signals is a good approach to integrate the necessary electronics into the suit. A completely different method is used for the commercial entertainment product of the company Microsoft called “Kinect”. It is a device that is able to capture the player’s motion. The underlying technology uses optical tracking system with one depth estimation camera. In combination with a game called “Dance Central” [Microsoft '11] a player can imitate dance movements shown by a virtual avatar. While doing this, he gets immediate visual feedback and some kind of evaluation, which tells him how close his movements were compared to the ones of the avatar. 3.3. Analysis and Motivation Georg Gerstweiler All previously mentioned projects have a close relation to this work. However these systems were developed for certain applications in the area of rehabilitation or sports. In many cases they are too specific to be used in another motion training area due to various reasons. The limitations refer to individual characteristics like the missing possibility to track arbitrary poses, a too sensitive hardware, a weak construction or the usage of devices which restrain the user’s movements. In many cases it is not even possible to interact with a second tracked object or a person. Even though human motion tracking is the main focus of an application, many approaches do hinder the user physically and therefore do not guarantee a high degree of realism. The effect of reduced mobility caused by the motion input device must not be underestimated, even if only using a prototype. Not only aspects like almost realistic animations for 3D characters or system latencies [Bideau '04] have an influence on the outcome of the project, but also the input devices play an important role. Some systems do not provide the ability of tracking all joints of interest over the whole time, for example due to a single viewing angle of the tracking system, and are therefore limited. Finally it has to be paid special attention to the user’s needs. Motion capturing suits equipped with relatively large devices could hurt the user, if he makes physical contact to hard materials like the floor or furniture. Many applications in martial arts or Related Work 22 Gerstweiler, Vonach, 2011 sporting activities force the user to lie respectively roll on the floor or require contact with other users. In addition to that some mentioned input devices like inertial sensors do not provide the possibility of interacting with a second tracked object, because of a missing global reference point. All these facts lead the authors of this thesis to the conclusion that there is a need for a full body motion tracking system, which is specialized for capturing a complex sequence of movements respectively teaching motion skills with adequate real time feedback. Additionally the focus of the system should lie on a more robust, but also more flexible and functional input device. Discussion of Requirements and Technologies Gerstweiler, Vonach, 2011 23 4. Discussion of Requirements and Technologies Georg Gerstweiler In order to analyze whether a VR setup is suitable for teaching motion skills it is significant to choose an optimal tracking technology for capturing complex physical body movements. But before any decision about this technology can be made, a wide range of requirements concerning technical specifications and also the usability in a motion tracking environment have to be considered. According to this discussion it should be possible to evaluate each of the tracking technologies which were introduced in chapter 3.1. Furthermore these requirements also draw a clear picture of the capabilities and characteristics the system should provide, especially in the context of an educational application. In addition to that the discussion in the following chapters should act as a guideline to help choosing respectively designing individual components, on the one hand for the tracking hardware and on the other hand also for the implementation of necessary software. The now following requirements concentrate on the aspects of practical use, necessary features and the educational field of application when running a motion capturing training session. 4.1. Requirements of a tracking system for teaching motion skills Georg Gerstweiler Since the tracking setup should provide the possibility of teaching different kinds of motion skills, a variety of considerations have been taken into account, in order to guarantee the best possible results. In general human body movements affect more than one human bone respectively joint. When talking about motion skills in this work it can be described as a complex chain of different body movements over a period of time, like people do while dancing or doing self-defense exercises. As a result the system should be capable of tracking as many human joints as possible at least down to wrist and ankle in a Discussion of Requirements and Technologies 24 Gerstweiler, Vonach, 2011 reasonable time. The more joints the tracking system is able to detect the more detailed motions can be captured and evaluated. Due to the fact that the whole system is intended to be used as a live training device, time plays a central role. The human brain needs about 15 updates per second in order to experience a sequence of frames as fluent. This update rate is also a minimum requirement to call an application real-time [Akenine-Möller '02]. The fact that people are nowadays used to see movies at a frame rate of about 24 fps the update rate should be higher than 15 fps. Depending on the speed of the captured body movement the optimal data update rate varies. Rapid exercises request higher rates to get reliable data, because a low sampling rate would falsify the records by dropping positions between two sequenced samples. This could lead to inaccurateness especially when analyzing movements. To guarantee a frame rate that allows to work in real- time, the tracking system has to detect each limb and calculate all joint angles of the human body in such a short period, that there is still enough time left to visualize the information. In addition to that it should be mentioned that a high frame rate also supports the effect of immersion in a virtual environment [Bowman '05]. For the user it should then be easier to experience the virtual training session. Not only the update rate but also the latency should allow the system to run in real-time. If this cannot be accomplished the user performance and the time he can work in such a VR environment decreases. Because if the human brain continuously does not get the visual feedback he would expect according to his experiences, due to a lag of time for example, there is a high risk of getting headache, nausea or dizziness. This condition is called “cyber sickness” and would force the user to quit the capturing session [Bowman '05]. Besides it ought to be considered that a training session should not be interrupted for 30 to 45 minutes by any physical or technical issue, so the user can fully concentrate on the exercise. During such a training session there should also be enough room, so that the user is able to perform the exercise freely without the need of avoiding any barriers. In addition to that the tracking system must be able to cover a volume which is big enough to detect every part of the body regardless of its position or Discussion of Requirements and Technologies Gerstweiler, Vonach, 2011 25 orientation. For example in case of performing martial arts or rehabilitation exercises a space of about three by three meters should be enough working area for one person. However when choosing an appropriate technology it has to be considered that a dancing simulation for example does require a bigger tracking volume. Because of that the chosen tracking technology should also be capable of expanding the working area. There are many tracking technologies that need devices mounted on the participant’s body. If such a system is used, these objects should not hinder or hurt the user when sitting or lying on them. For this reason mounted parts have to be chosen respectively constructed as small as possible, which means flat as well as flexible, however also robust enough to withstand an impact. To guarantee enough freedom the user has to be kept free of any wired connection between the body and the rest of the system. If any information has to be exchanged between the user and a server system, a wireless connection should be employed. Many motion tracking technologies especially with millimeter accuracy need some kind of device mounted directly on the user’s body. So it is likely that big parts of the body are covered by any material for holding these devices in place, such as clothing or straps. As a result of this it is necessary to think of the actual weight that must be carried and also of the hygienic aspect, due to the fact that there usually is more than just one user working with this system. In general the time that is needed before starting with the actual training session should be kept as short as possible in order to support the usability. If the training situation only involves certain regions of interest, like the head or the upper body, preferably it ought to be possible to track only these significant parts. In addition to that the tracking technology used for this environment should also work for people with different body dimensions or at least for a bigger group of people. Additionally it would be useful to track several objects respectively skeletons at the same time, for example the body movements of two people. To make this possible the system has to identify the different targets. So beside the orientation and global position of each joint also an identifier would be helpful to assign the tracked part to the right skeleton. In order to organize that huge amount of information created by two skeletons, some kind of synchronization between the tracking system and the tracked Discussion of Requirements and Technologies 26 Gerstweiler, Vonach, 2011 object could help. Especially a setting where two people are working collaboratively would benefit from this feature. However even if working with just one user it is still very likely to interact with a virtual object like a ball or some kind of tool. In general the used technology must also be able to deliver at least one global joint position in order to be able to calculate dependencies between the real and the virtual world. All the requirements mentioned above should now be helpful to decide which of the numerous available tracking technologies are suitable in this application area of teaching motion skills. 4.2. Tracking Technology Decision Georg Gerstweiler Since the tracking of the user’s movements is the basic element in this project, it is important to choose the most appropriate technology for this task in the context of learning motion skills. Any lack in quality of the data received from the tracking system would finally falsify the results. So a system is needed that delivers reliable positions and orientations of the human body, on the one hand in order to get viable data and on the other hand to robustly map the human movements to a virtual avatar. Based on the requirements mentioned in the last chapter some technologies are less suitable for this application area than others. The method of mechanical tracking for example would not be practical due to the complexity and size of the device, which would handicap the user while performing the exercises. Likewise neither the magnetic nor the acoustic tracking technology satisfies the demands. Magnetic tracking devices for example are not qualified due to their inadequate proportion between maximum distance and accuracy. The specifications of an acoustic tracking system show clearly that it would not be practical for this kind of usage, since one receiver is only able to deliver a position. Another two devices would be necessary to calculate the orientation of one limb. Additionally the update rate that is achievable with this technology would be too low to get a high rate of a continuous flow of data in real-time. In order get six dimensions of freedom and enough information for all required Discussion of Requirements and Technologies Gerstweiler, Vonach, 2011 27 body parts, a great number of transmitters respectively receivers would have to be attached to the body. These transmitters would have to send their signals in individual time slots, which would reduce the sampling rate to a level far below the required value. Inertial sensors however would perform pretty well considering update rate and distance. But since only relative positions are available, interaction with another tracked object respectively person would be difficult and inaccurate. In order to make this possible a second supplementary technology that is able to detect these objects in relation would be needed. Furthermore the inertial sensors still suffer on the constantly growing measurement error, which would make it hard to compare and evaluate motion tracking data sessions. Finally there is the optical tracking technology, which seems to be the most suitable one due to various reasons. Regarding this technology it must be distinguished between using markers or just natural features. The marker less concept with a depth estimation camera for example would satisfy almost all mentioned requirements. The main issue is that this technology is not yet fully developed. A single time-of-flight camera would work quite well as long as the user is standing directly facing towards the sensor. But workouts like self defense exercises and other comparable training situations do also require the user to turn around. With a single camera system this would result in a lot of occlusions, which makes it impossible to reliably track all movements. In addition to that these systems do either have resolutions of about 200 by 200 pixels and a millimeter accuracy like the CamCube 3.0 by PMD Technologies [PMDTechnologies '11] or have a higher resolution and therefore a much lower accuracy. Both approaches are not usable in this application area jet. The optical marker tracking system though would satisfy all the mentioned technical requirements, whereas only the outside-in approach can seriously be considered. This is because markers can be mounted on the human body much more unobtrusive in contrast to cameras. The following technical characteristics play an important role in order to get reliable data in a reasonable time. The update rate for example, which depends on the used camera system, is usually capturing with 30 Hz minimum and therefore high enough. Likewise the latency in such systems is in a range which allows it to run in real-time. In order to reliably compare two sets of motion data not only timing is important, but also Discussion of Requirements and Technologies 28 Gerstweiler, Vonach, 2011 accuracy. In an optical tracking system the accuracy is in the range of millimeters or even sub-millimeters, which meets the requirements. Furthermore if using a well calibrated multi-camera system the probability of object occlusion is reduced to a minimum. Finally the tracking volume, which depends on the number of cameras used, can be adjusted to the needs of a motion tracking system. In general there are two types of markers utilized in optical tracking systems: On the one hand passive markers and on the other hand active markers. In the area of optical motion tracking passive markers are usually made of small reflective balls. The benefit from this kind of markers is that the user does not need any power source or wiring on the body. So the motion suit mainly consists of tight clothes and markers stuck on them. But since any hardware jutting out from the body may be disturbing or even hurting the user, this approach would not be the perfect choice for a sportive application. An improvement would be to shrink the reflectors down to a flat patch attached on the body respectively on a suit. This however would have a negative effect on the angle of visibility of the markers. A possible alternative are active markers, which emit light that can be captured by cameras. Since the technology of LEDs has evolved concerning size and power consumption they are a good choice to be used as markers. Surface mounted LEDs for instance are very compact and small as well as robust. Therefore they can be attached very flatly on the suit to avoid hindering the user in any way. Furthermore the fact that there are LEDs with a high radiant intensity even at a viewing angle of up to almost 180 degrees makes it possible to stick them directly onto the surface of the suit without a significant effect on the angle of visibility. In addition to that LEDs make it possible to have more control over the markers. For example they could be synchronized with the tracking server to achieve a better object identification or even help calibrating a virtual skeleton. Considering all the so far mentioned facts, an optical tracking system with active LED markers should perform pretty well, in matters of getting reliable motion tracking data, which can then be evaluated and visualized. But there are still numerous challenges that have to be met to use this kind of motion tracking technology in an educational environment for teaching motion skills. In the first instance these challenges are concerning the design of a usable active motion Discussion of Requirements and Technologies Gerstweiler, Vonach, 2011 29 capturing suit. The idea of such a suit is closely related to the topic of wearable computing, where electronics and clothes are combined and should still be hygienic. All the components that are needed in order to realize this suit also have to be as unobtrusive as possible. Wires must not hang loose from the body, the mounting system of a marker should be as flat as possible, the power source ought to be small, but strong enough and finally all in all should not hinder the user in his freedom of motion. An active motion suit with LED markers is not a new concept and has already been used in other projects. For example Figure 10 shows a motion capturing shirt developed by J. A. Staples et al. [Staples '06]. Although it was not designed for sportive capturing sessions they paid attention not to hinder the user with the cabling and tried to hide all wires on the inside. The used active markers are made out of quite big pulse-driven LEDs with an additional reflector to produce a wide viewing angle. In sportive applications these big and jutting out devices could hinder the user or even break off while doing exercises. Many other projects do not pay much attention to robustness and flat surfaces. In Figure 11 active markers are shown which are developed by N. Kunju et al. for conducting kinematic measurements [Kunju '09] and Figure 12 visualizes a complete motion suit [Kirk '05] used in a project by A. G. Kirk. Both approaches use a very loose cabling. In addition to that the motion suit in Figure 12 is made out of many straps connected only with the conducting wires and seems to be very hard to put on correctly. All three mentioned examples are working with visible light LEDs. Nevertheless there is a high potential to improve the concept of LED markers in combination with an active motion capture suit for the use in an educational sportive VR environment. Figure 10: MoCap Shirt LED Markers [Staples '06] Figure 11: Active Markers for Kinematic Measurements [Kunju '09] Discussion of Requirements and Technologies 30 Gerstweiler, Vonach, 2011 Figure 12: Complete Active MoCap Suit [Kirk '05] 4.3. VR-Setup Georg Gerstweiler The previous chapter tried to point out which of the current available tracking technologies is suitable for the intended application area. Numerous arguments lead to an optical tracking technology that is based on active markers. It is now time to describe the details of the actually used equipment. The authors decided in favor of the “iotracker” [Pintaric '07], an infrared optical pose tracking system which meets all the mentioned requirements. It is a multi-camera tracking system originally designed to work with reflective markers. In order to deal with the problem of occlusions eight cameras are arranged at the corners of a cubical shaped training area. The used tracking volume has a quadratic base area with a lateral length of about four meters and a height of three meters. Because the system is normally using reflective markers it is now possible to turn off the IR flashes built around the sensors in order to use active infrared LED markers at the same wavelength of about 850 nm instead. This will also reduce unwanted reflections. The employed IEEE 1394a firewire cameras are working synchronized with a frame rate of 60 Hz at a resolution of 752 by 480 pixels. The iotracker software is programmed to estimate the position and orientation of so called “rigid-body targets”. These Discussion of Requirements and Technologies Gerstweiler, Vonach, 2011 31 consist of usually four markers each, mounted on a rigid construction of plastic rods in a specifically designed pattern to allow a better identification. The time it takes to calculate the vectors for six “degrees-of-freedom” varies between 18 and 40 milliseconds depending on the number of tracked rigid-body targets. The results of all detected targets are then distributed over the network for further processing. 4.4. Skeleton Estimation Algorithm Emanuel Vonach The iotracker system [Pintaric '07] described above is able to provide the position of all visible optical markers and for rigid-body targets the orientation as well. However in order to use this data for tasks like the animation of a virtual avatar, further information about the structure of the tracked objects respectively the relation of the markers to each other is required. This topology is usually referred to as “skeleton”, which includes joint positions, orientations and characteristics as well as the length of individual parts of the object (the limbs in case of a tracked person) and their hierarchic interrelations. In case of the project at hand, the employed algorithm to estimate the required skeleton is based on a paper by Kirk et al. [Kirk '05] and was further adapted by Schönauer [Schönauer '07]. The basic principles are the same: If a tracked object has an underlying articulated skeleton, this implies that the markers mounted on one segment of this object can only move together and with a constant distance to each other over time. Given this premise, it is possible to automatically find a skeleton that fits the movements of a tracked object (Figure 13a - d). This is accomplished in three stages. After observing the absolute distances between pairs of markers over time, in a first step it is possible to group those which move roughly together. In order to allow this, at least two markers on one segment and one on the adjacent are required. Since especially in optical tracking systems noisy data and occlusions have to be expected, better at least three markers should be attached to each limb for stable tracking. Discussion of Requirements and Technologies 32 Gerstweiler, Vonach, 2011 a. b. c. d. Figure 13: The estimation of a skeleton: a. one of the authors moving all his joints, b. the marker positions provided by the tracking system, c. the grouped markers, d. the fitting skeleton In the next stage the topology is determined by finding the joint positions that minimize the mean distance between the potential joints and each marker in the corresponding group. For this to be possible the markers have to be uniquely Discussion of Requirements and Technologies Gerstweiler, Vonach, 2011 33 identified and tracked over time. Although especially passive optical tracking systems do not relay identification information through the markers, they still can be labeled because they do not move far between two frames if the update rate of the tracking system is reasonable high. Still this information may be incomplete, which is why additional techniques for identification are employed. For example the facts that the distances between markers roughly stay the same or that similar configurations are likely to reappear in other frames can be used to calculate a likelihood of being the same marker. The third stage is optional and optimizes the lengths of the segments and offsets of the markers further. The major benefits of the described algorithm are amongst others that there is no need to perform any predefined movements or poses, neither to make some kind of manual measurements. It is only necessary that every joint of the tracked object or person is moved during a calibration phase, so that the system can estimate the underlying structure of the skeleton. This makes it also usable in settings where special poses or movements are impractical or impossible like in rehabilitation. The algorithm is relatively independent from any special marker layout as long as there are enough markers on each segment. In a normal motion capture session about 30 to 40 markers are employed for full body tracking. Nevertheless better results are achieved if the markers are not mounted too close to the joints and if they move as little as possible in relation to the corresponding skeleton segment. 4.5. Marker Layout Emanuel Vonach Basically the LED markers of the motion suit designed in this work do not have to be placed in accordance to a certain predefined pattern as long as certain elementary requirements are fulfilled, which are described later. This is due to the fact that the specific marker arrangements of each individual limb of a tracked person are automatically obtained by the tracking system in a separate calibration phase [Kirk '05, Schönauer '07], as described in the previous chapter. Nevertheless since it is almost impossible to change the marker Discussion of Requirements and Technologies 34 Gerstweiler, Vonach, 2011 positions after the construction, the authors have chosen the marker layout well- considered in order to make it compatible with other tracking algorithms as well. Individual necessary changes did not come up before the tracksuit as base material (see chapter 6.1.1) was selected; nevertheless they are already mentioned in this chapter for reasons of completeness. The marker pattern designed by the authors and described in detail below, is originally based upon a layout developed for the University of Ottawa as described in the Vicon Workstation Quick Reference Guide [Gordon '09] and the Marker Placement Guide of the Carnegie Mellon University Motion Capture Database [CMU '07]. The arrangement has been adapted and supplemented in order to meet the special requirements of the tracking algorithm used in this work’s setup. Accordingly there should be at least two to three markers for each limb and in contrast to the Vicon layout these should not necessarily be placed directly onto the skeletal joints [Schönauer '07]. Additionally the necessities which apply to the majority of optical tracking systems have been incorporated as well: The markers should be placed where they can be seen from a wide angle and their positions should change as little as possible in relation to the limb while the tracked person moves. In order to accomplish that it has proven effective to position the markers as close to the bone as possible or at least avoid regions with a lot of muscles. Furthermore it is beneficial to arrange the markers of the left and the right side of the body asymmetrically, in that way making it easier for any tracking system to distinguish them. In Figure 14 the final marker layout used in the construction of the motion suit is shown, summing up to a total of 70 markers. However the markers connected by a dashed line and labeled with a “+” are actually paired and can only be activated together. The authors decided on this approach to overcome certain constrictions of the used IC (chapter 6.2.1) since it is their judgment that these markers are usually to be used at the same time in most cases. Discussion of Requirements and Technologies Gerstweiler, Vonach, 2011 35 Figure 14: The marker layout for the motion suit. In the following the information written in parentheses always refers to Figure 14. Corresponding very much to the Vicon layout, the markers on the head are located one over each temple (H2 & H7) and ear canal (H3 & H6). Two markers are placed on the back of the head (H4 & H5), approximately in one plane with the markers over the temples but closer to each other. Another marker is attached somewhat higher, centrally on the forehead (H1). The shoulder markers (LA1 & RA1) are mounted right over the shoulder bone where they should be quite unaffected by arm movements. The same applies for the elbow marker pairs (LA4+ & RA4+) which are located on each end of the axis of the elbow joints. In contrast to the Vicon layout there is not just one marker on an outside line between shoulder and elbow, but two, placed with an offset from the mentioned line and asymmetrically shifted for the left and right arm (LA2 & LA3 respectively RA2 & RA3). They approximately divide the arms into thirds. Likewise there are also two markers instead of one on the forearm, Discussion of Requirements and Technologies 36 Gerstweiler, Vonach, 2011 one over the edge of the outer bone and the other one a little inwards from the middle of the arm (LA5 & LA6 and asymmetrically RA5 & RA6). The wrist markers are mounted in a similar way as the elbow markers directly over each end of the axis of the joints (LA7+ & RA7+) without any kind of bar or rod to offset the markers as it is used in the Vicon layout. In particular the layout for the hand has been adapted considerably. Instead of only one marker for each hand the authors decided to locate two markers over the back of the hand. One a few centimeters below the base of the index finger and the other at the base of the pinky finger, somewhat offset towards the outer edge of the hand (LA8 & LA9 respectively RA8 & RA9). The authors added a marker on the tip of each index finger as well in order to enable possible grip detection in the future (LA10 & RA10). The marker positions on the torso match the pattern proposed in the Vicon layout to a large extent. The front marker intended to be located between the two collarbones had to be moved down a few centimeters due to a zipper on the upper part of the tracksuit (T1). Another marker is placed at the lower end of the sternum (T2). On the back one marker is located over the base of the neck (T3) and another one somewhat down and to the right over the center of the scapula (T4). In addition to that in the Vicon layout a further marker (T5) is intended to be placed over the 10th vertebra, which cannot be accomplished with high accuracy in this case since the base material is a tracksuit and therefore the exact position is dependent on the body size of the user. Still the marker is located approximately over that region down the spine. The positions of the markers around the pelvis are only an approximation of the locations described in the Vicon layout for the same reason. Two markers meant to be located in front approximately where the pelvis bone juts out, had to be mounted a few centimeters down, above the lower seam of the upper clothing (T6 & T11) in order to make room for the wiring of the control circuitry. At the back there are another two markers closer to the center where the bone usually juts out a little as well (T8 & T9). The central marker on the sacral bone proposed in the Vicon layout as alternative to those two markers has been discarded. Furthermore two markers are positioned on the side of the body right upon the hip where one could imagine another axis for the pelvis (T7 & T10), although there is not really a joint. Discussion of Requirements and Technologies Gerstweiler, Vonach, 2011 37 The markers at the side of the buttocks are placed as far to the top of the thigh as possible (LL1 & RL1) so that they are just not covered by the upper part of the tracksuit. In case only the lower part is used they can be seen as upper boundary. The markers for knee (LL4+ & RL4+) and ankle (LF1+ & RF1+) are similarly to the elbow and wrist markers located at the end of the particular joint axis. Again there are two markers instead of the proposed one, dividing the upper and lower leg in thirds. On the upper leg one marker is placed on the far side of the thigh (LL3 respectively RL2) and the other one more to the front center (LL2 respectively RL3). Concerning the lower leg, one marker is positioned over the shin (LL5 respectively RL6) and the other one on the outer side (LL6 respectively RL5). The marker positions on the foot depend very much on the shoe size of the user. Two markers for each foot are meant to mark the imaginary joint axes between big and small toe (LF4+ respectively RF4+), another two should partition the region between ankle and toe markers in thirds and are intended to be located over the instep (LF2 & LF3 respectively RF2 & RF3). The last marker is destined for the heel bone (LF5 & RF5). The additional marker intended to be positioned on the tip of the big toe as described in Vicon layout has been omitted, because the toe is not rigid in relation to the remaining foot. Although it would be possible to track the toes as an individual limb, additional markers would be advisable in that case. Since tracking of the forefoot seems not essential and solid mounting of these markers would pose a challenge of its own the authors decided against it due to the unreasonable effort. Electronic Fundamentals and Communication Protocols 38 Gerstweiler, Vonach, 2011 5. Electronic Fundamentals and Communication Protocols Georg Gerstweiler Since this work is about designing an active motion suit it is important to understand the technology behind the active markers. This is particularly the case with “light emitting diodes” and the most common electronic wiring concept in connection with a large number of LEDs, namely the “matrix circuit”. A more basic introduction to electronic concepts can be found in appendix A. For the design of circuits including microcontrollers it is necessary to understand the communication between these electronic components. The specifications of such architectures called “protocols” are important, particularly their limitations. In the context of this thesis, the two employed standard protocols, the “Serial Peripheral Interface” and the “Inter-Integrated Circuit Protocol”, are described in the following. 5.1. Light Emitting Diode Emanuel Vonach The use of “light emitting diodes” is essential for this work, which is why in this chapter an introduction to the basic principles is presented, based on an explanation in [Harris '02]. Basically “diodes” are very simple semiconductors. One part of these devices consists of an insulator or other material with poor conductive quality. Within such a material the atomic bonding is so tight, that there are no free electrons which would allow a flow of electric current. For that reason in order to create a semiconductor this material is combined with traces of conductive matter. In this process either atoms with free electrons (N-type) or with free places for electrons (P-type material) are added. Both improve the conductivity of the original material as they allow electrons to move through the resulting semiconductor. Diodes consist of a conjoined N-type and a P-type section. In the original state the free electrons of the N-type section fill the free places in the adjacent P-type Electronic Fundamentals and Communication Protocols Gerstweiler, Vonach, 2011 39 material (Figure 15a), creating an insulating region called depletion zone (Figure 15b), which prevents the flow of current. If the side of the N-type material is connected to the negative end of a circuit, the P-type side to the positive end and a high enough voltage is provided, free electrons of the N-type region push those in the depletion zone out of their places. These electrons on their part push other electrons out of their places, and so on. In this way the depletion zone dissipates and current can flow through the diode (Figure 15c). Connected with reversed polarity, the free electrons would just accumulate on one end of the semiconductor and the free places on the other one, resulting in an even bigger depletion zone. This is why diodes only conduct current in one direction. a. b. c. Figure 15: Schematic of a diode: without power supply a. electrons fill the holes and b. create a depletion zone, c. if a power supply is connected this zone dissipates. Electronic Fundamentals and Communication Protocols 40 Gerstweiler, Vonach, 2011 Jumping from atom to atom within the semiconductor, the electrons release energy in the form of light. The amount of energy released depends on the material of the semiconductor and correlates with the wavelength and therewith the color of the emitted light. In ordinary diodes most of the energy is immediately absorbed by the semiconductor material itself. However so-called “light emitting diodes”, in short LEDs, are specially designed to emit the light outwards. Usually a plastic bulb helps to concentrate the light. Especially in recent years the application areas for LEDs have got rapidly more manifold. They are now found in a diversity of devices ranging from everyday items like digital clocks over traffic lights to huge LED displays. 5.2. LED Matrix Circuit Emanuel Vonach In a number of applications it is desirable to individually switch on and off a larger number of LEDs. Every day examples for such a case are a digital watch or an LED ticker. A rudimentary solution for this challenge would be to address every diode separately, in which case usually a driver chip for every four to eight devices would be required. With a growing number of LEDs it would get a complex task to synchronize those chips, not to mention the unnecessary high power consumption. This is why according to [Jordan '02] it is common practice to use a technique, namely “time division multiplexing”. In this way not all LEDs are switched on simultaneously, but rapidly in sequence in groups of usually four to eight. Due to the relatively small temporal resolution of the human eye it looks as if all light sources are active at the same time. This effect is known as persistence of vision and also accounts for a camera set to a frame rate below the frequency in which the assembly is cycled through. In order to implement this approach, the emitters are organized in rows and columns to a so-called “LED Matrix”. The cathodes of all diodes in a row are connected to each other, as well as all the anodes of a column. A special IC called “LED Matrix Driver” makes sure that in every time slot only one row is actually connected to ground while the rest is switched to high resistance or positive electrical potential. At the same time positive voltage is applied to those Electronic Fundamentals and Communication Protocols Gerstweiler, Vonach, 2011 41 columns of LEDs which are designated to be active in this time slot (Figure 16). The rest is consistently connected either to high resistance or a negative potential. Since diodes allow current only to flow in one direction only those LEDs are part of a closed circuit where positive potential is applied at the column and negative potential or ground is connected to the corresponding row in the same time slot. Figure 16: Only the LEDs of a matrix circuit that are connected to positive potential and ground in the same time slot are active. 5.3. Galvanic Coupling Emanuel Vonach In the event that two or more individual circuits are sharing lines, for example a common ground or voltage source, a delicate issue called “galvanic” or “common impedance coupling” may arise [Langguth '06]. If the common impedance is large enough, for instance due to long wires, poor conductivity or an ohmic resistance, the circuits can disturb each other through superimposed voltages. This can result amongst others in an unwanted voltage drop at the common impedance (Figure 17). Particularly for AC applications it has to be pointed out that the impedance of the shared line is consisting of a resistive as well as of an inductive component. For that reason the disturbing voltage is not only dependent on the current, but Electronic Fundamentals and Communication Protocols 42 Gerstweiler, Vonach, 2011 also on its rate of change. Especially at high frequencies its influence exceeds the one of the resistive part by far. The simplest approach to minimize disturbances caused by galvanic coupling is to avoid connections between individual circuits as far as possible. If common lines are unavoidable, length, resistivity and self-inductance should be kept at a minimum. Figure 17: Due to the galvanic coupling between circuits I and II, the voltage uC drops at the common impedance ZC and a signal U2 may be disturbed. 5.4. Serial Peripheral Interface – SPI Georg Gerstweiler The “Serial Peripheral Interface” as described in [Freescale '10] and [Leens '09] is a serial bus technology that was developed to establish a synchronous interface between integrated circuits such as microcontrollers and on-board peripheral devices. It was invented 1979 by the semiconductor branch of Motorola which was outsourced and renamed to Freescale in 2004 [Freescale '10]. The basic concept behind SPI is a communication between one central device, the “master”, and one or more parallel connected “slave” devices over a system of four wires which are SLKC, MOSI, MISO and SS (Figure 18). SLCK describes the wire for the clock signal which is connected to each slave. This signal is generated in the master device in order to synchronize the data which is transported over two different wires, one for each direction. These outputs are called “Master Out Slave In” (MOSI) respectively “Master In Slave Out” (MISO) and enable a full duplex communication. The only wire that needs to be an individual direct connection to every single slave is the “Slave Select” (SS) line. Electronic Fundamentals and Communication Protocols Gerstweiler, Vonach, 2011 43 Figure 18: SPI Functional Principle Before establishing a connection a couple of parameters have to be set in a special 8-bit control register situated in the master device. The most important values within this register are SPIE which enables interrupt requests, SPE which specifies the connection of the four ports to the right system functions and MSTR which defines the device as master. CPOL and CPHA configure the SPI clock for one of four possible communication modes. CPOL is responsible for the clock polarity and CPHA defines at which edge of the clock signal the data is sampled. Before it is possible to send bits over the two data lines to a specific slave the corresponding SS line has to be set to low which enables the slave and the clock signal. In Figure 19 an example communication is described, which is sampling the data on the rising clock edge while the bits at the data lines (MOSI and MISO) are toggled on the falling edge. Figure 19: SPI Communication Sample [Maxim '09] Electronic Fundamentals and Communication Protocols 44 Gerstweiler, Vonach, 2011 The main elements [Freescale '10] for communicating over the data wires are two 8-bit data registers. One register is located on the master, the other one on the slave device. Both are connected by the MOSI (DIN in Figure 19) and MISO (DOUT in Figure 19) wires. This concept forms a 16-bit shift register. The communication itself is based on shifting the 16-bit register eight positions at a time to exchange the data between the master and the slave device. That makes it possible to send commands and to receive data at the same time. After a complete transmission the data is moved into a separate register, the receive data register, and is kept there until a new 8-bit word arrives. This register is also used to buffer data until it is shifted over the data line. The specification of SPI is kept simple. There is no definition about the maximum data rate, an addressing scheme or any acknowledgement mechanism. For that reason a high level protocol is required. 5.5. Inter-Integrated Circuit – I²C Emanuel Vonach Originally developed in 1982, according to [Leens '09] and [Irazabal '03] the “Inter-Integrated Circuit Protocol” (short IIC or more commonly I²C) utilizes only two bus lines to connect an almost unlimited number of devices, each distinguished by a unique address. The actual data transfer only takes place between a single “master” and one “slave”. The two involved bus lines are called “Serial Data” (SDA) and “Serial Clock” (SCL). Both can be pulled by resistors either to the positive supply (usually +3.3 V or +5.5 V, recently also +2.5 V), that way representing a logic high or “1”, or to ground meaning a logic low or “0”. Generally logic on the data bus is only allowed to be changed while the clock signal is low. Only special “start” and “stop” conditions are an exception, as they both occur while SCL is high. A transition from high to low state is recognized as “start”, analogically the contrary means “stop”. Any IC can start a communication as long as there is no other data transfer in progress. The device generates a “start” condition, thereby assuming the role of a “master” and declaring the bus busy. All other devices at that time are considered as “slaves” and listen. In the case of two devices trying to initiate a Electronic Fundamentals and Communication Protocols Gerstweiler, Vonach, 2011 45 communication at the same time, eventually one writes logic low while the other tries to write logic high. The structure of the bus prevents a short circuit, thus only the logic low is actually transmitted. Since a master also checks what it sends against what it can read from the bus at the same time, the IC notices a conflict and has to wait until the bus is free again. During a data transfer the master generates a clock signal on the SCL which is used by all other devices to synchronize bus readout. If a slave has problems to keep up with the given clock speed, the above-mentioned bus structure allows it to hold the clock signal low until it is ready. The master on the other hand notices the delay and has to wait with the next cycle until SCL is released. Typically the first seven bits of a message contain the unique address of the designated receiver. Thereof the first four bits are fixed, while the last three can be set by hardware pins, for example if similar devices are to be used at the same time. The use of 10-bit addresses is also possible. The eighth bit of the message specifies whether the master wants to write (logic low) or read data (logic high). By now the designated target of the message has recognized its address and signals its readiness by pulling the SDA line to logic low during the ninth clock cycle. This is understood as an “acknowledge” signal. In that way a master is also able to check whether an optional IC is connected to the bus or is not. After that the sender (either master or slave, depending on the read/write bit) writes the desired number of 8-bit data words, waiting for an “acknowledge” signal from the receiver after each word. Once the data transfer is complete, the master finally generates a “stop” condition, signaling to all other ICs that the bus is free again. For an overview of the I²C communication structure see Figure 20. The I²C protocol supports data rates of 100 kbit/s in “standard” mode, 400 kbit/s in “fast” mode and up to 3.4 Mbit/s in “high-speed” mode. It is reasonable robust concerning noise and provides collision detection and arbitration for multiple masters. Today a large and manifold number of devices either offer I²C ports or can easily emulate the protocol in software. Electronic Fundamentals and Communication Protocols 46 Gerstweiler, Vonach, 2011 NXP and Freescale are in charge of the official specifications. Although the implementation of the protocol is free of licensing fees, officially allocating I²C addresses is not. Figure 20: I²C Communication Procedure Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 47 6. Motion Suit - Conceptual Design Georg Gerstweiler Figure 21: Motion Suit Material Overview The requirements described previously, now lead to the conceptual design of the motion suit. In the now following chapters the three important topics hardware, software and testing are discussed, which have to be clarified before the selected respectively self-developed components can be assembled in order to get a fully functional motion input device. The chapters “material” and “control box components” are dealing with fundamental pieces of hardware, Motion Suit - Conceptual Design 48 Gerstweiler, Vonach, 2011 which should on the one hand be compatible among each other and on the other hand allow all required functions mentioned in chapter 4.1. These components not only include electronic hardware, but also fabric materials like cloths. Figure 21 illustrates a brief overview of the main materials in this chapter. Only after the final decision concerning the used hardware has been made it is possible to design a detailed software architecture to get a clear impression of how the individual software and hardware components are intended to interact with each other. 6.1. Material Georg Gerstweiler This first part of the conceptual design chapter is principally concerned with components, which are used to build up the motion suit, but are not part of the control circuitry. According to the requirements and after extensive testing the most suitable active and passive electronic components, but also materials made out of cloth, are carefully selected. 6.1.1. Base Material Emanuel Vonach Although the choice of the base material for the motion suit could seem secondary, it is an important decision due to various reasons. The selected material has to support the placement of the active markers onto the tracked person in a way that they reliably stay on the intended positions even when the person is moving. Nevertheless the motion of the user must not be influenced or hindered by the base material in any way. On the contrary it should be unobtrusive and light, but still robust and flexible. In addition to that it has to be considered that the material might cover large parts of the user’s body and should therefore be highly breathable in order to minimize transpiration. In any case the hygienic aspect has to be considered: Either the supporting material should be wearable above the regular clothing, or in case the markers are mounted onto some kind of garment, it has to be washable. In addition to that it Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 49 also would be preferable if the system can be used by persons with different body sizes. There are several ways to deal with this challenge: One possibility would be a system based on straps. In that way only small areas of the wearer’s body are covered by the material and he would not have to sweat more than usual. Besides such a system could be worn above regular clothing which would avoid hygienic issues. Nevertheless the authors decided against this design because systems of that kind are usually wearisome to put on and are therefore not very user-friendly. It would be questionable if every wearer could put it on properly without help. In order to avoid getting out of place, the straps would have to be fastened very tight but would still have to be flexible enough to ensure freedom of motion. Powering the active markers would also be an issue, because there would not be much space to place the wiring onto the straps. In the worst case this could result in the need for loose wires, which should be avoided as far as possible. As an alternative the authors decided for a tracksuit as base material. Since these suits are designed for sports anyway they are generally light, breathable and robust as well as washable. In contrast to straps the user should be able to put it on right intuitively. The authors chose a very tight but stretchable model to avoid the markers getting out of place and still be able to accommodate a wide range of body sizes. In order to avoid retraction of the sleeves and to allow the placement of the markers on the back of the hand and finger the suit has to be improved. Similar enhancements have to be devised for the trouser leg respectively the ankle. A headband is suitable for positioning markers on the head. However a wired connection between the headband and the upper cloth of the suit will not be avoidable without unreasonable effort. For the placement of markers on the feet the authors decided for an extension of straps which should be wearable above normal footgear, in that way circumventing hygienic issues as well as the problem of different shoe sizes. It would also be advisable to choose a suit in a color which is absorbing light in the used spectrum and which has no reflector bands in order to avoid false matches by the tracking system. Usually black or dark dyes are a good choice. A tracksuit produced by “Newline” and a headband by “Prosport” were finally Motion Suit - Conceptual Design 50 Gerstweiler, Vonach, 2011 selected (Figure 22), both bought in a store for sporting goods. The shirt and trousers were chosen in a medium size, so that a tight fit for users of different body dimensions is possible. In case of the headband one size fits all anyway. The particular concepts the authors came up with to meet the described challenges are described in chapter 7.2. Figure 22: This track suit and headband are a suitable base material for the motion suit. 6.1.2. Conductive Thread Georg Gerstweiler The active markers used in this system have to be continuously supplied with current. Since the markers are spread all over the suit an appropriate wiring technology has to be chosen. Although it would be convenient to use wires on top of the suit in order to connect the active markers with the power source, the risk of hindering the user with loose cables or breaking a wire during a practice session would be high. Because there will be 70 markers spread all over the body, the wiring can get very complex and finding and repairing a broken circuit could take a lot of time. Furthermore all the wires would have to be removable for washing the suit. Another method for powering the LEDs could be to use one battery per marker. This however would be possible due to the low power Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 51 consumption of LEDs, but it is far away from being practical. It would take some time to turn all markers on or off and since infrared markers are used, finding an empty battery or switching on all markers could be quite a challenge. In addition to that it would not be possible to address each marker individually. A thread though has some considerable advantages, which make it almost perfect for being used as conductor. In Figure 23 a spool of conductive thread is shown, which can be used for powering the active markers. One spool of this distributor [Smith '11] is delivered with a length of 200 yards and is made of three thin twisted threads each coated with a micron layer of natural silver. This gives the thread its conductivity and a dark gold and silver appearance. According to its description it is actually meant for repairing fencing suits [Smith '11]. The main advantage of a thread in contrast to a wire is that it does not break no matter how often it is bended or buckled. So it should be robust enough to be used for the motion suit. Furthermore a sewed flexible thread will not hinder any movement of the user, because the wiring is interlaced into the suit. Also fast movements that put a lot of strain onto the thread will not easily cause it to be torn apart, since it showed a relatively high tear-resistance. When doing exercises the users will sweat and therefore it will be necessary to clean the whole suit from time to time. If a conductive thread is used however, it is possible to wash the whole suit without removing all the wiring. On the other hand the design of the sewing layout in a way that short circuits are prevented will take plenty of preparation in the construction phase. Figure 23: Spool of Conductive Thread Figure 24: This graph shows the conductivity of the thread measured with a digital circuit analyzer, when using multiple threads for the same connection over a distance of 10 cm. Motion Suit - Conceptual Design 52 Gerstweiler, Vonach, 2011 Suitability tests conducted by the authors showed, that the electronic characteristics of the conductive thread have a weakness in contrast to an ordinary wire. A commonly used copper conductor has a resistance that can be disregarded in closed electronic circuits over short distances. The conductive thread however showed a resistance of about 0.5 Ω to 1 Ω per centimeter during measurements. This has a great influence on the design of the circuit, considering the distances which have to be covered. The paths even get longer, due to the sewing method, because the thread has to be applied in a zig-zag-pattern to keep the fabric elastic. However a good balance between resistance and the width of the zig-zag has to be found. Furthermore measurements and tests conducted by the authors showed that it is beneficial to sew every path at least three times. This reduces the resistance by about two-thirds as shown in Figure 24. Any further effort would not lead to a significantly better result. 6.1.3. Insulation Emanuel Vonach If the conductive thread ran exposed and uncovered all over the suit, the risk of different circuits getting crossed accidentally due to movements of the user would be very high. As a result the behavior of the active markers would be unreliable and they could light unintentionally or with reduced brightness. In the worst case a short circuit could occur and might even damage the hardware. For those reasons the insulation of the conductive paths is essential. A first and important step is the physical separation of the electric potentials. In order to ensure this, all seams of conductive thread transporting positive current can be sewed on the inside of the suit and the ones for negative current on the outside. However to keep the wires located on the same side of the suit apart from each other and for further protection against corrosion, an insulation has to be applied. The material employed for this task has to meet a number of preconditions. Besides providing excellent insulating properties, it must not be brittle but keep a certain amount of flexibility after application, because otherwise it would break and flake of with time. Furthermore it has to be robust, Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 53 washable and obviously compatible with the base material and the conductive thread. Especially the conductive properties of the latter must not be negatively affected. The relatively large-area application as well as the fact that the skin of the user might repeatedly be in direct contact with the insulating material also have to be considered. In order to investigate the properties of different potential insulation materials and to assess the suitability for the intended task, the authors sewed a number of seams of conductive thread with a specific length onto a piece of cloth as a test environment (Figure 25a). Since there is no product on the market especially designed for this kind of application area, a variety of materials, most notably black paint for model making, silicone as well as different fabric glues and paints, was applied. Subsequently the characteristics were analyzed, especially the insulating efficiency, conductivity of the thread and the elasticity. After sufficient drying neither of the paints showed reliable insulating capabilities, not even the ones that get puffy after application. Silicone turned out to be an outstanding insulator and also remained very flexible, but for one thing it does not feel comfortable on skin and for another it is not robust enough and would peel off with time. Common fabric glue also provided quite convincing results, but feels unpleasantly scraping on the skin. Furthermore the application to large areas might be difficult since vast quantities of glue would be required, making this approach inefficient and expensive. Nevertheless a related product proved to be ideal, not just regarding robustness and remaining flexibility, but it also feels acceptable on the skin: “Textiltransfer-Gel” by CreaDirect GmbH [CreaDirect '11] (Figure 25b), a kind of fabric glue especially designed for a craft called “napkin decoupage”. This technique allows common napkins to be used as decoration on a variety of things. Since this particular product is designed for large-area usage on clothes, it is cheaper and can easily be applied with a brush. After 24 hours of drying the coating can be carefully fixated with an electric iron on a very low setting, which makes it washable with up to 30° C according to the manual. Motion Suit - Conceptual Design 54 Gerstweiler, Vonach, 2011 6.1.4. Connectors Emanuel Vonach Actually it would be best to omit connectors as far as possible, but in some cases they cannot be avoided. The interconnections between headband, shirt and trousers have to be accomplished using some kind of connectors. Furthermore an electrical control circuit will be required and a wired connection to an operating station is out of the question. Since it would be unreasonable difficult to construct a washable control circuit, it has to be removable as well. The employed connectors should be as small as possible so that they do not get in the way, but at the same time strong enough so that they do not unplug accidentally. Additionally the connectors mounted on the cloth have to be either resistant to corrosion or removable to keep the resulting suit washable. The authors decided for the model “Micro MATE-N-LOK 3 mm” by Tyco Electronics [TE '07] for the connection between the separate pieces of clothing (Figure 26a). The crimp contacts allow the direct conjunction with the conductive thread and are plated with a tin alloy which makes them corrosion- resistant. The housing of the connectors consists of nylon and is fitted with a locking mechanism. Altogether this connector should be robust enough to withstand the expected level of stress but still has the right dimensions for comfortable handling while being unobtrusive nonetheless. In order to connect the control circuit to the suit, a connector with a very high pin count is needed. The model chosen above is only available with up to 24 contacts, which will not be enough. Even if a version with enough pins was a. b. Figure 25: a. Test Setup for Insulation Material, b. Glue for Napkin Decoupage Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 55 available, it would then be too big to be applicable since the centerline spacing is 3 mm. A good alternative the authors came up with to merge such a high number of wires on a small space are simple ribbon cable connectors, also known as “Insulation-Displacement Connectors” (IDC) with DIN 41651. The authors selected a 40-way “Wiremount Socket” of 3M’s “3000 Series” [3M '11] (Figure 26b) and a matching counterpart, namely a “LAT-CON Series 057” low profile IDC header by ITW Pancon [ITW] (Figure 26c). Special attention was paid to select models with gold and nickel plated contacts so that they are non- corroding. The crossover from the conductive thread to a ribbon cable has to be dealt with during the construction phase (see chapter 7.4). The connection to the actual control circuit board can be managed with a “SEK 18, P.C.B. Transition” by Harting [Harting]. Another challenge to be met is the mounting and connection of the active markers on the suit. Despite intensive research the authors could not find such small LED sockets as required for the intended application area. And even if the diodes would probably withstand washing, it is unadvisable to fixate them directly upon the suit, because there should also be the possibility to change individual LEDs without great effort. That is the reason why the authors have to create a suitable socket on their own. a. b. c. Figure 26: a. “Micro MATE-N-LOK”, b. “Wiremount Socket” and c. IDC Header with Ribbon Cable 6.1.5. IR-Emitter Georg Gerstweiler In chapter 5.1 the functionality of light emitting diodes was already described. For designing an active motion suit, an appropriate device acting as an active Motion Suit - Conceptual Design 56 Gerstweiler, Vonach, 2011 marker has to be chosen. However infrared light emitting diodes seem to be a good choice in this use case. Since there are numerous different infrared LEDs on the market it is possible to select a type that perfectly meets the requirements of the motion suit respectively the application area of an educational training system. There are four main groups of requirements that have to be considered: the technical aspects of motion tracking, electronic concerns, educational requirements of sports as well as the user’s health. This chapter describes why the “HSDL-4400-1L1-IR” [HewlettPackard] emitter is an adequate choice for this application. Regarding the educational requirements of teaching motion skills it is important, that all the devices which are mounted directly on the motion suit are as small as possible so that they do not hinder the user in his movements. In addition to that these devices must also be robust to a certain extent to withstand at least a light hit. All these characteristics lead to the SMD (Surface Mounted Device) version of the HSDL-4400 unit (see Figure 27), which is usually mounted on printed circuit boards and therefore has a compact cubical design with a lateral length of only 2 mm. Figure 27: SMD Version of the IR-LED HSDL-4400 In the first place the VR environment is intended to be used with a four to eight camera system and a maximum distance between two cameras of five meters. For that reason the used LED model has to be bright enough to be captured by a camera over at least this distance. In addition to that the marker has to cover a big enough region on the camera sensor in order to be recognized. The digital representation of a marker is called “blob”. Since LEDs have a more directional radiant intensity compared to common light bulbs, it is significant that the emitter can also be identified by the tracking algorithm, even if it does not point directly at the lens of a camera. Therefore not only the brightness at the centerline, but also at high angles has to be considered. Figure 28 illustrates the radiant intensity in contrast to the angular displacement of the chosen LED. According to this graph the HSDL-4400 still emits 50 % of its maximum intensity Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 57 at an angle of 55°. This is made possible by the special lens on top of the diode. The distributer states that an “untinted, nondiffused truncated lense” [HewlettPackard] is used to achieve this characteristic. Figure 28: Relative Radiant Intensity vs. Angular Displacement [HewlettPackard] Figure 29: Peak Forward Current vs. Forward Voltage [HewlettPackard] The electronic challenge of using active markers draws considerable attention to the power source respectively the power consumption. Because of the fact that the motion suit has 70 active markers attached and a high series resistance to each LED due to the use of conductive thread, it is necessary to balance the marker intensity with the power dissipation. It should be aimed for only consuming as much energy as necessary, since the power source also has to be placed on the motion suit and should therefore be preferably small and light. Figure 29 illustrates the behavior of the diode regarding voltage and current. In order to choose the right values for powering the markers, Figure 30 compares an image of a passive reflective marker with several pictures of an infrared LED. The passive reflective marker (Figure 30 a) is native to the iotracker [Pintaric '07] and has a diameter of 1.6 cm. The Figure 30 b - i shows the HSDL-4400 with increasing power levels from left to right. In addition to that the first row represents an emitter pointed directly towards the camera, whereas the images of the second row are captured from a side view with the emitter rotated by 90°. All images have been taken at a distance of five meters with the same camera model that is used in the iotracker system. In all images the iotracker recognized a valid blob. This demonstrates that the HSDL-4400 is bright enough even at the lowest tested power supply of about 1.3 V and Motion Suit - Conceptual Design 58 Gerstweiler, Vonach, 2011 10 mA. In order to get good results the evident difference in the blob size between f and g suggests that the minimum power supply should be at 1.4 V and 20 mA, which corresponds to a power dissipation of about 28 mW per LED. passive marker LED at ~13 mW LED at ~28 mW LED at ~56 mW LED at ~96 mW a. di re ct ly t ow ar ds c am b. c. d. e. ro ta te d by 9 0° f. g. h. i. Figure 30: Comparison between a. passive reflective and b.-i. active IR-LED markers at a distance of 5 m. Due to the fact that infrared light cannot be seen by humans it is important to know if long exposure times of high intensity emitters can lead to injuries. The “Commission for European Electrotechnical Standardization” (CENELEC) issues safety regulations for working with electronic devices. According to the datasheet the commission cleared the HSDL-4400 “regulated for eye safety” within the EN60825-1 [HewlettPackard]. 6.1.6. Emission Regulation Emanuel Vonach Especially in the near vicinity of the markers considerable reflections have to be expected due to the high radiant intensity of the LEDs. The dye of the suit can only absorb a part of them. As a result the employed optical sensors would very likely capture distorted images of the LEDs and the used tracking algorithm might produce false or imprecise data. It might even detect reflections as false markers or wrongly dismiss genuine markers if for example their roundness is used as differentiating feature. Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 59 For those reasons the authors decided to use an additional material to regulate the emissions of the LEDs. It has to absorb infrared light well and should be light, flexible and easy to apply. Accordingly the authors tested a variety of different materials and textiles for these properties (Figure 31a). They were especially focusing on the infrared reflectance since a lot of dark dyes only absorb light in the visible spectrum. The best results have been achieved with black foam craft sheets, available in common handicraft shops. They are easy to work with and absorb infrared light very well, which is why they can be used to construct pads for emission regulation around each LED (Figure 31b). They are also very soft and therefore additionally work as shock absorbers. a. b. Figure 31: a. Test Setup in a Furniture Store, b. Possible Emission Regulation Pad 6.2. Control Box Components Georg Gerstweiler The control box houses the central logical circuitry, which is responsible for communication and for controlling the motion suit. In this chapter the two involved ICs and their functions in the system are described. In addition to that the very important aspect of the total power consumption is discussed in order to be able to choose an adequate power source. The final part of this chapter will present the smaller components, which are necessary to build a functional control box. With the results from the previous sections and the hardware Motion Suit - Conceptual Design 60 Gerstweiler, Vonach, 2011 components presented in the following, the authors are finally able to actually construct a fully operational active motion capture suit. 6.2.1. LED Matrix Driver Emanuel Vonach In order to be able to benefit from the advantages of an active marker system to full capacity, it is not enough to just power the LEDs, but they also have to be individually controllable. For one thing it is necessary to provide at least 20 mA and 1.4 V per LED, as explained in chapter 6.1.5. Though this value has to be regarded as absolute minimum for reliable tracking and it would be preferable, if the active markers could be powered with at least 40 mA to be on the safe side. After all, dimming them if they are too bright is much easier accomplished than increasing the current if the available source cannot provide more in the first place. It should also be possible to comfortably switch on and off individual LEDs or to assign predefined marker patterns. Furthermore it would be beneficial to be able to synchronize certain marker arrangements per frame with the tracking system. For these tasks special ICs can be used, so-called “LED drivers”. The authors considered a number of possible concepts of how to realize the marker control. Common LED drivers are designed to operate only individual or small groups of diodes. Such devices can provide a relatively high current, but in order to control a high number of markers, several drivers would be required as well as an additional IC for coordination. This would result in a complex circuitry and eventually lead to a larger control box than desired. Especially for high numbers of LEDs it is reasonable to organize them in a matrix (see chapter 5.2) and use a special “LED Matrix-” or “LED Display Driver”. These drivers are normally employed to handle so-called “Seven-Segment Displays”, for example commonly known from pocket calculators. These displays can usually show several digits, built up of seven individual segments plus a decimal point each. Since such applications typically need only a relatively small current it is almost impossible to find a model capable of providing at least 40 mA per LED. Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 61 a. b. Figure 32: a. Photo and b. Schematic of the MAX6954 LED Matrix Driver by Maxim [Maxim '09] After extensive research the authors decided in favor of the “MAX6954” by Maxim [Maxim '09]. They selected the 36-Pin SSOP (Shrink Small-Outline Package), since this version is especially small, which helps to keep the control box compact (Figure 32a). The chip is able to control up to 16 Seven-Segment displays with additional decimal point, 128 discrete LEDs or other configurations, organized as a common cathode LED Matrix with up to 16 digits (rows) and 8 segments (columns). The pattern to be displayed has to be set in a register in static RAM for each digit. The internal multiplex circuitry scans along the digits, powering each segment with typically 40 mA for a maximum of 15/16th of a cycle. The maximum possible current is stated with 46.5 mA, but not recommended. The outputs O0 to O18 of the driver (see Figure 32b) sequentially source current to the anodes of the display, whereas depending on the time slot one of the outputs O0 to O7 works as sink for the current from the common cathode. In each time slot two outputs are inactive and set to high impedance during that time. The peak segment current is determined by an external resistor directly connecting ISET (Figure 32b) to the ground. For example a 56 kOhm resistor Motion Suit - Conceptual Design 62 Gerstweiler, Vonach, 2011 regulates the maximum current to the recommended value of 40 mA, a higher resistance would result in a lower peak current. The exact formulae to calculate the dimensions of components like this can be found in the datasheet [Maxim '09]. In addition to that the intensity can be modified further by setting a special register to adjust the duty cycle for each digit from 1/16th to 15/16th of a timeslot. By specifying a scan limit, which describes how much digits are active, the relative brightness is influenced as well, since it determines indirectly the length of a timeslot. The overall update rate of the display can be set from 1 to 8 MHz either through an external clock or by connecting a capacitor from the matrix driver pin OSC (see Figure 32b) to ground, but is also depending on the resistor which regulates the intensity. For example if the resistance at ISET is 56 kOhm, then a 22 pF capacitor at OSC sets the update rate to 4 MHz, which is by far enough for the suggested 30 Hz of a real time application. Concerning the power supply the display driver can operate with a supply voltage of at least 2.7 V up to 5.5 V. Nevertheless in order to minimize the power dissipation the voltage source should be chosen adequately considering the voltage drop of the driver of minimum 0.8 V, plus the ones of the LED and the conductive thread connected to one active output in one timeslot. This can be calculated with Ohm’s Law (chapter A.5). The driver has a typical supply current of 22 mA and a maximum of 35 mA, depending on a lot of factors like temperature, scan limit, etc, plus the total current of all segments active at the same time. For electronic decoupling of the power supply it is recommended to place a 0.1 uF and a 47 uF capacitor directly between anode and cathode of the power source and as close to the IC as possible. For communication the MAX6954 offers a 26 MHz SPI compatible interface (see chapter 5.4). In order to allow also data transmissions from host ICs working with a lower supply voltage than the driver itself, the SPI ports (Chip- Select Input CS, Serial-Data Output DOUT, Serial-Clock Input CLK and Serial- Data Input DIN, see Figure 32b) are designed to work with low logical thresholds of 0.6 V and 1.8 V. For this reason it is suggested, that these ports are connected to ground either through 4.7 kOhm resistors or 33 pF capacitors. Other interesting features of the display driver include five configurable general purpose ports, a “test mode” which forces all LEDs on with a duty cycle of 7/16th independent from the current configuration, a “blink mode” which automatically Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 63 flips between two separate segment patterns, and much more. If desired, several display drivers can also be daisy chained by connecting the SPI ports DOUT and DIN. Preliminary tests with an oscilloscope showed that the driver is suitable to provide the desired 40 mA per LED even with all eight scan lines active, but then the length of a timeslot is only 0.25 ms. This means the effective brightness captured by the used sensors is actually much dimmer. In order to compensate for the loss of brightness the authors decided to limit multiplexing to only four scan lines to reduce the time between active cycles. Still the effects of the multiplexing cannot be completely foreseen and require further tests (see chapter 7.1). 6.2.2. Controlling Unit Georg Gerstweiler The control box requires a logic circuit that must be able to process instructions from a PC and to control the LED matrix driver. In addition to that it has to provide several communication methods. The LED driver demands an SPI interface (see chapter 5.4), which communicates over four wires. Furthermore it is planned to integrate a device that can receive a trigger signal (see chapter 6.3) over a wireless connection. The latter device will be addressed via I²C (for more information see chapter 5.5). In addition to that the size and weight of the control box have to be considered. These requirements lead towards a microcontroller, which has small dimensions and low power consumption, but is still sophisticated enough to handle all required tasks. But in order to upload code and transmit commands to the IC, a communication interface like USB would be necessary. That feature requires besides a USB connector also an FTDI chip, which handles the serial connection between a PC and an IC. All this functions and much more are combined in a group of development kits, called “Arduino” [Arduino '11]. This fast open source prototyping platform provides hardware and software components, which enable a lot of functionality. The advantage of using such an Arduino board instead of an individual microcontroller is that in case of the Arduino the IC is preconfigured Motion Suit - Conceptual Design 64 Gerstweiler, Vonach, 2011 with a “bootloader”. This is a small program which handles the data arriving at the FTDI chip and enables the upload of code without any additional hardware like a so called “programmer”. The Arduino also provides a protocol for serial communication to a PC during runtime. The various Arduino boards are ready- assembled and differ in size and also slightly in the specifications. Regarding the mentioned requirements the “Arduino Nano” (see Figure 33) is the best solution for this project. With a size of about 18 mm by 44 mm it is the smallest available version with an FTDI chip and a type Mini-B USB connector on board. Figure 33: Arduino Nano In general all the available boards make use of a microcontroller from Atmel which is part of the 8-bit ATmega family. The Arduino Nano in the version 2.3, which the authors selected for the motion suit control box, uses the “ATmega168”. This standard 8-bit microcontroller provides several analog and digital interface pins, which can be defined for data input as well as for data output. It operates at a clock speed of 16 MHz. The microcontroller itself needs an input voltage of 2.7 - 5.5 V, whereas the Arduino as a whole requires at least 7 V from an external voltage source in order to operate reliably. It is also possible to power the board over a USB connection. According to the datasheet of the ATmega168, it just needs about 13 mA when running on 16 MHz. The actual power consumption of the whole Arduino will be definitely higher, because it also depends on the IC load. For that reason it will be measured during the first practical tests described later in chapter 7.1. Another interesting aspect of the specification deals with the different types of storage available on the chip. The ATmega168 provides 16 KB of flash memory, which is used to store the uploaded program. Furthermore there is also a permanent memory (EEPROM) with a capacity of exactly 512 bytes, which can be used to store and load the state of the program for example to Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 65 restore the system state if the power supply is disconnected. Finally there is 1 KB of volatile memory (SRAM) integrated as working memory. 6.2.3. Power Source and Consumption Georg Gerstweiler A main criterion for choosing components of the active motion suit is a low power consumption. Due to the fact that the suit is intended to be used wirelessly, the active markers and the control box have to be powered by a mobile voltage source. This power supply should be selected carefully, because size and weight increase with the capacity. In order to be able to pick an appropriate one, it is necessary to estimate the total power consumption including the control box and the LED markers in advance. There are three different kinds of devices that need a relevant amount of electric power through a continuous power supply during a tracking session. These are the Arduino Nano [Arduino '11], the MAX6954 LED driver [Maxim '09] and finally the HSDL-4400 high powered infrared LEDs [HewlettPackard]. Unfortunately there is no declaration in the Arduino Nano specification list that could be used to draw any conclusions about the typical power consumption. Only the recommended input voltage is declared with 7 - 12 V and according to the schematic the IC is supplied with 5 V. The datasheet of the incorporated ATmega168 however estimates a usage of 13 mA, when connected to a potential difference of 5 V. Measurements conducted by the authors showed that the power consumption of the whole board draws about 50 mA of current from a 7.4 V power source, while the ATmega168 was just busy with the bootloader but nothing else. This results in a minimum consumption of 0.37 W. The second active device located in the control box is the MAX6954. It can be supplied with a voltage of 2.7 - 5.5 V. Since the authors expect a very high resistance between the LED driver and the markers at least 5 V have to be provided to compensate for the voltage drop. The actual power dissipation of this device has to be calculated with a formula described in the datasheet [Maxim '09], which also includes the connected LEDs. As discussed in previous chapters the HSDL-4400 should be supplied with the maximum power the LED driver supports. So each LED is intended to run at a forward voltage of about Motion Suit - Conceptual Design 66 Gerstweiler, Vonach, 2011 1.4 V and a current of 40 mA. These values in combination with the maximum duty cycle and the highest possible number of active markers per output of the LED driver result in an absolute peak power dissipation of 4.4 W. All together there will be an approximate power consumption of 4.8 W. Since the Arduino Nano needs the greatest potential difference of at least 7 V, a mobile voltage source that is able to provide this minimum value and 4.8 W for about one hour should be chosen. Several different technologies for mobile power sources exist. The most convenient one for this kind of application area is a lithium-ion accumulator, which provides a good tradeoff between size, weight and capacity. In order to supply the recommended 7 V a two cell version is required, since one cell only provides about 3.7 V which sums up to 7.4 V. Previous calculations showed a total consumption of 4.8 W for the whole motion suit, which now results in combination with the electric potential of the power source in a flow of about 650 mA. Thus every accumulator with a capacity of more than 700 mAh should theoretically power the motion suit for at least one hour. The authors decided to use a lithium-ion polymer accumulator with 1200 mAh at 7.4 V, with a size of 67 mm by 31 mm by 15 mm and a weight of just 61 g. 6.2.4. Additional Components Emanuel Vonach Beside the crucial components mentioned up to now, there are also a lot of minor parts. These components are still indispensable for constructing a working motion suit, which is why they are subsequently summarized. All parts of the motion suit circuitry are relatively tolerant concerning the input voltage and can also work with a very small current, which is why precautions have to be taken to prevent the accumulators from deep discharge. The authors dismissed the possibility to incorporate a battery guard which would automatically switch off the system if the accumulator power gets too low, because in their opinion it would be more practicable to let the user finish a training session in that case. Therefore another device is used to allow this, namely the “LiPo Saver” by Modelcraft (Figure 34a), which is designed for two Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 67 celled accumulators. This simple yet effective component lights a red warning LED if the supply voltage drops below a threshold of 6.3 V. Since the Arduino Nano [Arduino '11] requires a different operating voltage than the MAX6954 display driver and in order to keep the voltage level stable over the whole operating time, another vital component is used, a so-called “voltage regulator”. Although the Arduino Nano has one built in, the powering of the display driver has to be done separately, because the built-in regulator is not able to handle the necessary currents. The “L78S00 Positive Voltage Regulator” by SGS-Thomson [SGS '93] (Figure 34b) can provide the desired stable output voltage of 5 V at an input voltage of up to 35 V and 2 A. Other required auxiliary components are standard aluminum electrolytic or ceramic capacitors of different capacities to ensure decoupling of the power source to avoid possible disruptive fluctuations. In order to level out an expected variability of resistances due to the varying lengths of the wire to the active markers, the authors intend to employ extra small SMD resistors. For activation and deactivation of the control box a preferably small toggle switch (Figure 34c) which can withstand at least 1.5 A will be provided. To allow the user to see if the suit is active as well as to give minimal feedback, an additional green LED is planned. Finally a standard power jack should allow connecting either an accumulator or optionally an external power source, as long as the voltage does not lie outside the range of ideally 7 to 12 V or maximal 6 to 20 V. a. b. c. Figure 34: a. “LiPo Saver” by Modelcraft, b. Voltage Regulator, c. Toggle Switch 6.3. Software Architecture for Operation and Management Georg Gerstweiler The previous chapters described components that will be used to assemble the motion suit. A number of them are part of the already mentioned control box, Motion Suit - Conceptual Design 68 Gerstweiler, Vonach, 2011 which has two main tasks. On the one hand it manages the operation of the markers and on the other hand it is responsible for the communication with other parts of the system like a PC. Thus a well-considered architecture is required to structure the collaboration between the individual modules, which will be presented in the following. So far only the hardware of the motion suit control box was introduced, but apart from that there are software modules needed as well. Additionally it is necessary to implement customized communication protocols in order to enable the whole functionality of the motion suit. Figure 35: Motion Suit Control Architecture The central device in this architecture is the motion suit control box. It contains two important units. On the one hand the integrated circuit ATmega168 and on the other hand the LED driver MAX6954 [Maxim '09]. The ATmega is built into the Arduino Nano v.2.3 [Arduino '11] and is used as the central controlling and communication component in this system. In order to have control over each Motion Suit - Conceptual Design Gerstweiler, Vonach, 2011 69 LED marker the IC is instructing the MAX6954 LED driver, which is able to address each marker of the motion suit individually. The advantage of this driver chip lies in the matrix circuit. This wiring method uses much less connections between the markers and the driver unit than for example a setup that is using several microcontrollers instead of an LED driver. The MAX6954 is able to support up to 128 LEDs (16 LEDs in 8 rows). In order to control the LED driver the ATmega168 is activating the desired marker pattern by sending its commands over the Serial Peripheral Interface. To support the tracking system in the identification of different body parts or objects it is useful to switch between different such marker patterns in sequence within a short time period. In order to speed up the time it takes to load a specific marker arrangement, they already have to be predefined. These so called “frames” store the status of each marker of the motion suit. The ATmega168 has a fast but volatile storage available in form of an SRAM, as well as a slower but permanent EEPROM. The first one can be used to hold a look up table (LUT), which defines an identification number for several frames as well as the appropriate status and configuration of the markers. Since the content of the volatile memory is lost if the power is disconnected, this information should be permanently synchronized with the EEPROM. In order to make use of this feature to full extent the IC should provide an interface for receiving an external trigger. This signal only has to address a specific frame number and thereby triggers the corresponding marker configuration precisely at the desired moment. Because the tracking system works with cameras that capture at a frame rate of up to 60 Hz, the communication architecture that transports the trigger signal should be fast enough so that a change of the marker setting can be accomplished within one frame. In this context the Inter-Integrated Circuit Protocol (I²C) can be used, since the speed limit of 3.4 Mbit/s in High Speed Mode is enough to transport an identification number in time [Irazabal '03]. Given that the trigger signal is intended to be generated by the tracking system in further development steps, the transmission of this signal should be accomplished over a wireless technology. However it has to be chosen carefully, because many technologies have a big latency or a heavy protocol Motion Suit - Conceptual Design 70 Gerstweiler, Vonach, 2011 load which would slow down the motion suit update rate. A wireless solution with these specifications could be a simple radio frequency module. The last branch in this architecture deals with a user interface application that has to make it possible to define the previous mentioned frames on a work station. Such software enables the user to configure all required settings including the marker pattern and the LED intensity. Furthermore it has to be capable of communicating with the control box over a serial interface. In addition to the mentioned features a reliable communication protocol has to be designed to enable the user interface software to upload and update the marker configurations and settings in the look up table. Another custom protocol has to be implemented to structure the data, which is sent with the trigger signal. Motion Suit – Construction Gerstweiler, Vonach, 2011 71 7. Motion Suit – Construction Emanuel Vonach As first stage of the actual construction phase, the feasibility of the devised concepts presented in the previous chapters is confirmed. After that the practical applications of those ideas as implemented by the authors are described in detail. Special attention will not only be paid to the interconnections and the interrelations of the individual components, but also to the necessary supplementations of the purchased material and particularly to the authors’ own developments in context of this work. The required adaptations of the tracking suit will be described, which were necessary to convert it into a suitable base material for a motion capture suit. Then the realization of the wiring and the interconnection of the individual pieces of clothing and the control box as well as the consequential challenges are shown. Finally the construction of the active markers is presented, followed by the assembly of the control box as well as the closely interrelated implementation of the motion suit software for management and operation. 7.1. Feasibility Tests Georg Gerstweiler This chapter describes preliminary tests that were performed before assembling the motion suit in order to be sure that the prepared concepts actually work. These tests focused on the cooperation between the Arduino and the LED driver to prove that it is possible to have an individual control over each LED and still a marker intensity that is bright enough to be recognized by the tracking system. The concepts and the components used for these setups have already been described in the previous chapters. With the following three steps it was possible to build a fully functional prototype. The first test environment was intended to verify whether or not an LED display driver is an adequate device to control the markers for the motion suit. In its standard application area this device is used to operate a display with multiple segments in a closed environment, not individual LEDs spread over a wide Motion Suit – Construction 72 Gerstweiler, Vonach, 2011 area. The first assembly to test the driver’s capabilities is visualized in Figure 36. Due to the fact that it is impossible to see infrared light, for this setup the IR emitters were replaced with standard 5 mm red, green and yellow LEDs, which emit light in a for humans visible spectrum and thereby allow a direct analysis without any technical aids. These emitters need voltages between 2 - 2.5 V and a maximum of 20 mA of current. As a control unit the Arduino NG [Arduino '11] was used, which has almost the same specifications as the Arduino Nano [Arduino '11] (see chapter 6.2.2) but is easier to work with because of its larger dimensions and its simply connectable socket contacts. In this early arrangement a MAX7219 [Maxim '03] in a 24-pin DIP (Dual In-Line Package) was responsible for driving the LEDs. It has comparable specifications to the MAX6954 [Maxim '09] and also the communication over SPI works in the same way. Although it is only able to drive eight digits with eight segments at a maximum of 40 mA per digit, the basic principle is similar and the MAX7219 can be used more easily without soldering due to its DIP profile. Figure 36: First test setup with Arduino NG, MAX7219 and standard 5 mm LEDs. For the test all components were assembled on a breadboard to build up a prototype. They were power supplied via a USB connection on the Arduino. That is possible, because in this test all the devices together do not need more than 500 mA, which every USB controller is able to provide. This includes an Motion Suit – Construction Gerstweiler, Vonach, 2011 73 eight by eight matrix of LEDs, which was also arranged on the breadboard in order to simulate the same situation as on the motion suit. To supply the emitters with the maximum current the LED driver is capable of, the corresponding resistor had to be connected to the input pin ISET. In addition to that every line of the matrix was equipped with a resistor to simulate the conductive thread that is intended to be used on the actual motion suit. This setup was already equipped with a status LED to support debugging steps. Primarily this test was planned to confirm several devised concepts concerning the control of an LED driver and the correct addressing of the emitters. First of all it should prove the compatibility of the Arduino board with an LED matrix driver. For that reason the SPI interface was initialized and tested. It was then possible to write and read from special registers located on the MAX7219. In the next step the accurately assembled LED matrix circuit was used to assess the individual marker control. Because the MAX7219 only forwards 40 mA for a whole row it was only possible to instruct the driver to enable two out of eight LEDs per row with the required 20 mA per LED. With the test matrix it was then possible to power two arbitrary LEDs in each row seemingly at the same time. Furthermore it should be analyzed whether time multiplexing has a negative effect on the appearance of the LEDs. The assembly was therefore captured by a camera with 60 fps which corresponds to a shutter speed that is slow enough to allow the matrix driver to scan the whole LED matrix multiple times (1300 Hz). This means that every captured image shows all LEDs of the whole matrix as if they were continuously supplied with power. Because of the time multiplexing the actual exposure time of one marker is just a fraction of the total exposure time. As a result the image of the marker appears darker than with a continuous power supply of 20 mA. This leads to the conclusion that an LED driver is needed, which provides more power in order to reach a comparable captured brightness as during the tests with the IR-LEDs described in chapter 6.1.5. Looking at the setup in Figure 36 it gives a clear picture of how complex the task of designing the paths on the motion suit can get. While running the test it turned out that too many crossing wires caused unexpected behavior like turning on the wrong LEDs, which probably happened because of interfering electro-magnetic fields. Motion Suit – Construction 74 Gerstweiler, Vonach, 2011 When constructing the motion suit later, the authors paid special attention not to get signals on the wrong lines. The observed issues also showed that capacitors are essential to stabilize the power supply, thereby eliminating current fluctuations, which could also cause interfering signals. Since the first setup confirmed the functionality of the planned concepts, another test with the final hardware used for the motion suit could be performed. Because the MAX7219 cannot provide enough power to supply the HSDL-4400 infrared LED with the required current, in the second assembly (see Figure 37) the MAX6954 is used [Maxim '09]. This model is able to provide 40 mA per marker, which should be enough to counteract the negative side effect of the time multiplexing of the matrix driver. Figure 37: Second test setup with Arduino Nano, MAX6954, HSDL-4400 and a resistance between 20 Ω and 150 Ω in series with each LED. Instead of the Arduino NG the Arduino Nano v2.3 is included that is also to be used for the construction of actual the motion suit. However before working with these devices in the test environment, it was necessary to solder the very compact SSOP (Shrink Small-Outline Package) version of the MAX6954 on a standard SSOP to DIP (Dual Inline Package) adaptor board to increase the raster size. In addition to that, the 5 mm LEDs were replaced with HSDL-4400 IR-LEDs. Since the MAX6954 has the ability to drive 8 rows with 16 columns it is possible to activate 16 LEDs in one timeslot. The main advantage of having more columns in the matrix is that fewer lines can be used, which increases the duration a single emitter is supplied with current and therefore it appears to be Motion Suit – Construction Gerstweiler, Vonach, 2011 75 brighter on the camera image. This on the on hand raises the power consumption, but on the other hand makes it feasible to adapt the marker intensity in a wider range and to reach the desired level of brightness. For that reason the authors decided to use just four rows and all available 16 columns of the matrix. Because the marker layout was planned with a total of 70 markers this limitation makes it necessary to combine some LEDs with low priority. As described in chapter 4.5 the markers which are mounted directly over the joints will not be controllable individually but in pairs. This results in the necessity of controlling 60 individual markers. In the test setup each infrared emitter was connected in series with a different resistor between 20 Ω and 150 Ω to find out the largest tolerable resistance respectively divergence regarding the blob size. Figure 38 shows the iotracker camera image of two lines with 16 emitters each, however the LED driver is set to display four lines in order to have the same scan rate and therefore the same effective brightness as with all 70 markers connected. Only in the right upper corner where the resistance is above 150 Ω a significant drop in the intensity can be recognized, whereas there is almost no visual difference where resistors between 20 Ω and 100 Ω were used. This is because the LED driver is supplied with potential difference of 5 V and 150 Ω cause a voltage drop that raises the total required voltage above the available 5 V. In order to calculate the maximum resistance which would not affect the intensity all consuming devices in this branch of the circuit have to be considered. The available 5 V can be divided among the LED driver, which needs 0.8 V and the LED, which requires about 1.5 V. The remaining 2.7 V would be absorbed by the LED driver itself, if there would be no additional series resistance. According to Ohm’s law the maximum resistance without losing intensity depends on the previously mentioned remaining voltage and the current of 40 mA that is needed for powering the HSDL-4400. This means that the conductive thread could cause 67.5 Ω without affecting the marker intensity. However this test also showed that the LED driver still operates reliably, if there are series resistances above this limit. Nevertheless the wiring on the motion suit should not considerably exceed the resistance of 67.5 Ω. Motion Suit – Construction 76 Gerstweiler, Vonach, 2011 Figure 38: Iotracker camera image of second test setup. The last test setups proved that the selected devices and concepts concerning the control box are functional and can be realized. Before working on the actual motion suit there had to be a final prototype, which should confirm that the used base material and the conductive thread are working well together with the control circuitry. Before building this prototype all planned marker positions of the motion suit were tested with the suit described in chapter 6.1.1. Therefore small pieces of hook-and-loop tape were temporarily stitched onto the clothing at the locations intended for the LEDs so that passive markers could be attached. Then the whole layout was successfully tested with the iotracker. After fine tuning, the final marker positions were fixed and a first prototype of a headband was constructed (see Figure 39) with the planned seven markers for head tracking. During the construction of this prototype the authors faced multiple challenges concerning the combination of cloth and electronic. Some of the concepts used in this stage are actually a lookahead, which is why they will be described in detail in the following chapters in the appropriate context. It was possible to permanently connect the conductive thread with metal contacts through a technique using heat shrinkable tubing and glue. Furthermore a first mounting structure was developed in order to link the poles of the LEDs with the wires. Obviously it is necessary to isolate the anode from the cathode all over the suit. To achieve this, the poles were separated through the fabric itself. In this setup the ground thread was sewed on the inside and the positive one on the outside of the headband. All LEDs were connected according to the schematic described in the matrix circuit chapter 5.2, which means that there was a common ground and an individual supply line for each emitter. All wires lead to a connector, which could be plugged into a counterpart mounted on the upper body. In this setup the connector was directly linked with the corresponding I/O ports of the LED driver. Motion Suit – Construction Gerstweiler, Vonach, 2011 77 When powering the LEDs it turned out that they did not shine as bright as expected. The reason for this was found in the wiring method. The conductive paths are galvanically coupled due to the common ground and therefore the current of all seven emitters, a total of 280 mA, had to flow to the MAX6954 over a relatively high resistance. This fact causes an immense voltage drop of about 8 V at the common conductor. As a result of this each LED was fitted with two direct connections to the control box to overcome this problem when constructing the motion suit. Figure 39: First prototype on clothing with connector and HSDL-4400. This final step concluded the feasibility tests since it proved that the authors’ concepts for building the motion suit are reasonable. However all these tests also pointed out some issues which had to be considered when constructing the actual motion suit, for example the wiring problem, the maximum resistance or the interfering currents. The insights acquired during the feasibility tests were very helpful for the actual construction, described in the following. 7.2. Suit Adaptations Emanuel Vonach As already anticipated during the conceptual design phase, the chosen tracksuit used as base material had to be modified and extended at certain points in order to match the requirements. Although a medium dress size had been Motion Suit – Construction 78 Gerstweiler, Vonach, 2011 selected so that the suit fits as tight as possible, a few adaptations were necessary to hold all parts of the clothing in the right place. First of all, to keep the sleeves from retracting and to maintain tension, the authors attached a loop out of a 15x200 mm elastic webbing band at the sides of every sleeve to be slipped over the index fingers. In order to increase comfort and to enable the placement of a marker at the fingertip, the tip and the back of the index finger was cut out of an ordinary glove and sewed onto each of the loops as it can be seen in Figure 40. Figure 40: Adaptation for the Hand A similar difficulty had to be overcome concerning the trouser legs since they should maintain tension as well. For that reason another 40x210 mm loop of elastic webbing was sewed at the sides of every trouser leg so that it can be fixated around the heel. In this context also the challenge of how to place active markers on the feet had to be met. After considering a number of approaches, the authors decided for an extension of straps which the user can wear over his own shoes. The structure described in the following consists of three major parts and is illustrated in Figure 41a - c and Figure 42. The first component, a 90x150x170 mm trapeze piece of cloth, was sewed on the lower end of each of the suit’s trouser legs in the area of the heel (with an overlap of 55 mm) and complemented with the loop part of a 30x145 mm hook-and-loop tape at the bottom end (Figure 41a). The second component was attached about the instep: an 80x225 mm rectangular strap of cloth combined with two straps (55x320 mm and 55x260 mm) stitched crossways Motion Suit – Construction Gerstweiler, Vonach, 2011 79 (Figure 41b). 30x100 mm of the hook part of hook-and-loop tape was sewed at each of the ends of these straps. a. b. c. Figure 41: a. The heel piece, b. the instep piece and c. the strap to fixate them on the foot. The last component consists of two 45x350 mm straps with a cap of 70x185 mm in between and two 30x320 mm loop tapes sewed on the outside (Figure 41c). On the inside another two hook-and-loop tapes are affixed, but about half of each (the piece closer to the heel) is hook tape and the other half loop tape. The resulting component can be spanned from the users tip of the shoe to the heel, where the both ends can be fixated on each other. In that way it is then possible to affix the trapeze heel piece as well as the straps from the instep there and both are held in place (Figure 42). Figure 42: Strap Construction for the Feet Motion Suit – Construction 80 Gerstweiler, Vonach, 2011 In order to ensure that the trousers and the upper clothing are kept together and stay in place, the authors added four loops to the trousers around the hip, two on the front and two on the back, to be put through slots cut out of the shirt (Figure 43). These loops are made out of an 50x80 mm (50x100 mm on the back) loop part of a hook-and-loop tape attached onto the trousers and 50x35 mm of the hook part which was sewed over the end of the tape. The slots in the upper clothing measure 70x5 mm each. They were especially reinforced with 65x15 mm of the loop part of hook-and-loop tape, stitched upside down onto the cloth. The sides of the slots were strengthened with 15x15 mm pieces of tape as well, cut in about two thirds. As it can be seen in Figure 44, in order to give the top additional solidity in that region and to hold it in place in case the trousers are not used, a cord has been thread in the lower seam which can be knotted over the belly. Figure 43: Four of these loops are mounted on the trousers and can be pulled through slots in the shirt. Figure 44: The cord can be knotted over the belly for additional solidity. Motion Suit – Construction Gerstweiler, Vonach, 2011 81 7.3. Wiring Emanuel Vonach Perhaps one of the most basic yet crucial questions about building an active motion capture suit is how to transport the power to the markers. The circuits have to allow a sufficient flow of current, but at the same time must not limit the flexibility of the suit or limit the user in his movements. Moreover the control circuitry has to be located in a place where it does not get in the way or hurt the user. The authors decided that the most suitable location to accomplish this is on the belly, which means that all the wiring has to be routed to this position eventually. These challenges as well as others, maybe less obvious ones, had to be met. In order to eliminate the possibility of a short circuit due to colliding limbs of the user, the authors decided to physically separate the electronic potentials as far as possible. For that purpose all the conductive thread transporting positive current was sewed on the inside of the suit and the one for negative current on the outside. In that way even if the user accidentally crosses different circuits, they are either connected to ports of the LED driver which are inactive in this timeslot or the current is simply divided between several ports. On the inside the user himself respectively his clothes act as additional insulator preventing unintentional connections between circuits. Furthermore to be on the safe side and to minimize corrosion, all conductive seams were insulated with fabric glue which was carefully applied with a brush and finished with a flat iron. Since the conductive thread has a considerable ohmic resistance especially with longer distances, each seam had to be stitched multiple times to increase the conductor cross-section and thereby countering this effect. In that manner the longest connections were sewed up to four times and in a few individual cases the conductivity was additionally supplemented with conductive silver paint. Nevertheless issues like galvanic coupling or an excessive voltage drop could occur if several active markers shared a common connection with high resistivity as shown in the feasibility tests. Increasing the conductor cross-section further would almost eliminate the flexibility of the thread, which is why the wiring for each LED had to be done separately as far as possible. This Motion Suit – Construction 82 Gerstweiler, Vonach, 2011 resulted in a substantial number of parallel conductive paths. But since the control box is located at the belly, especially in the chest, waist and upper thigh regions there would have been so many parallel seams required that it would have lead to a severely reduced flexibility if they had been sewed directly on the suit. For example up to 60 parallel connections would have had to be applied within an area of about 90 cm across the chest and back. In any case the separate connections of all the planned 60 LEDs have to converge within a preferably small region around the waist, where they have to be routed into the control box. In the authors’ opinion accomplishing this without sacrificing too much of the suit’s flexibility or even destroying it seemed an impossible task. For that reason bridgeovers, also called “highways” by the authors, made of elastic webbing band were designed for the especially delicate regions (as illustrated in Figure 45a - c). For the upper part of the trousers two bands of elastic webbing sized 60x225 mm and 60x205 mm for each thigh were equipped with parallel seams of conductive thread in a distance of 6 mm to each other, up to 10 mm from each end of the webbing (see Figure 45a). As it can be seen in the figure, around one end the seams were stitched slightly closer together to allow more compact sets of cables later. It should also be noted that the webbing has stretched about one or two centimeters after stitching and before further assembly an insulating layer of fabric glue was applied. The two bands of the bridgeover were put on top of each other on the inside of the suit, both facing with the conductive side away from the fabric of the trousers (see Figure 45b). Now the conductive threads sewed on the outside of the suit were pulled through the cloth and connected with the shorter inner band, while the ones sewed on the inside were routed to the longer piece of webbing on top. At the upper end of the thigh all conductors, a total of 18 contacts per leg plus 4 from the active markers not connected to the bridgeovers, were finally pulled to the outside of the suit in two parallel rows (Figure 45c). The highways were only fixated with the ends on the suit to ensure a high degree of flexibility. Motion Suit – Construction Gerstweiler, Vonach, 2011 83 a. b. c. Figure 45: a. Bridgeovers with Parallel Seams, b. Applied Insulation, c. Contacts Pulled to the Outside Figure 46: Bridgeovers for the Upper Body In a similar way three highways for the upper body were constructed for a total of 60 contacts. For the circuitry of both arms a 60x580 mm and a 60x470 mm band of webbing for each side was used as well as another 60x630 mm and 60x570 mm for the conductors of the headband, chest and back. The distance of the parallel seams of conductive thread on the highways is 5 mm. In order to make connecting the bridgeovers and the control box respectively the arms easier, the seams on the highways of the left and right side of the body end diagonally: each individual seam goes 7 mm further than the one before (see Motion Suit – Construction 84 Gerstweiler, Vonach, 2011 Figure 46). The highway for the headband and the rest of the upper body ends in a triangular shape. The conductive seams of the suit were connected to the highways as described before and insulated with fabric glue. Finally all threads were pulled through the cloth to the outside in parallel rows (respectively triangles) in the region around the belly. In order to connect such a high number of contacts to the highways, frequent crossings of the conductive thread of the outer and inner seams in a very small region were unavoidable. Tests conducted by the authors had shown that these intersections could lead to unintended short circuits because of the thin fabric of the tracking suit and the repeated stitching. For that reason additional patches of cloth were designed to reinforce these critical regions and ensure insulation to provide safe crossing areas (see Figure 47). Figure 47: Patch to Reinforce Regions with Multiple Crossings As already mentioned the markers at the joints are directly linked together. For merging these circuits a similar technique as for the highways was used, but since in each case there are already other circuits running through between the two LEDs, the method had to be adapted. Depending on the location of the bridgeovers, a 30 mm wide elastic webbing band of suitable length was fitted up to the half with two parallel seams of conductive thread with a distance of 10 mm (see Figure 48a). Then the anode and cathode of both LEDs in question were connected to these seams. To ensure the electrical insulation the other half of the webbing was folded and stitched directly on top (Figure 48b). In the middle of the bridgeovers the conductive thread was pulled to the appropriate Motion Suit – Construction Gerstweiler, Vonach, 2011 85 side of the suit again and sewed further from there. Additionally the elastic webbing of these connections helps to hold the markers in place. a. b. Figure 48: Bridgeover for Joint LEDs a. Before and b. After Mounting 7.4. Connectors and Connections Emanuel Vonach After addressing the challenge of the wiring there still remains the question of how to interlink the individual parts of the motion suit as well as how to manage the junction of the conductive thread with the connectors and the control box. In case of the headband, all the circuits were pulled to the outside at the back of the head in two parallel rows. In order to create cables to interconnect the headband with the top of the suit, the authors first made knots with all threads belonging to the same conductive path to make sure that they stay in place. There are several threads for each path because of the multiple stitching as mentioned before. Each knot was additionally fixated with a drop of superglue before the loose ends were cut to an appropriate length. After that a sufficient amount of heat shrinkable tubing was prepared in lengths from 250 to 280 mm (in pairs with 5 mm steps), which should be long enough to allow unhindered Motion Suit – Construction 86 Gerstweiler, Vonach, 2011 head movements, but not so long that it would get in the way (Figure 49 shows the same technique applied on the trousers). Starting from the right hand side the pieces of tubing were slipped over the bundles of conductive thread with the help of a thin piece of wire formed to a hook. Then it was shrunk with a flat iron on a low to medium setting, but not higher to avoid damaging the fabric. Afterwards the conductive threads of each cable were knotted together at the ends of the tubing and then tied onto the socket contact of the connector. Again each knot was additionally fixated with superglue. Finally the contact was closed tight around the knot with pliers and pushed into the connector housing until it locked in place. Figure 49: Cables of Conductive Thread with Heat Shrinkable Tubing as Insulation The technique described above was also applied for the corresponding connector on the shoulder. The two parallel rows of conductive thread from the designated highway plus the ends of a few additional seams from the top of the suit were fitted with heat shrinkable tubing of about 10 to 30 mm and then incorporated into the connector accordingly. The same applies for the cables from the trousers (Figure 49), with the only difference that the sockets for the plugs were directly built into the control box. The bundles of conductive thread were pulled from the highway to the outside of the suit in two rows, 130 mm from the upper edge of the trousers so that there is enough space for the overlapping top of the suit. Again a few additional circuits from near markers Motion Suit – Construction Gerstweiler, Vonach, 2011 87 were sewed there as well. The length of the used heat shrinkable tubing was stepped between 290 and 310 mm from the center outwards. All the tubing for the upper row was cut another 10 mm shorter. As already explained before, the convergence of such a high number of individual circuits to the control box as required for the motion suit posed a special challenge. This is because it had to be accomplished within a comparatively small area around the belly and an inconvenience for the user had to be avoided as far as possible while still keeping everything robust. In the authors’ opinion the best way to meet this challenge was using ribbon cables for the connection to the control box, which means that a method had to be devised to manage the transition between conductive thread and the stranded wires of the ribbon cable, illustrated in Figure 50a - e. For that purpose the authors had divided the 60 contacts from the highways as well as the 16 seams from the rest of the upper body evenly to the left, the right and the center of the belly. Due to the limited space, four less were routed to the center though. After that four pieces of 10-way ribbon cable with a length of 80 mm each were clamped into a 40-pin IDC for both sides. More precisely one piece of cable was always clamped alternately to the left and to right into the connector, starting on the left for the left one and vice versa (Figure 50a). Afterwards the individual conductors of the ribbon cables were separated and cut into the right length with a box cutter so that they match the pattern of the conductive thread bundles to which they are connected to (Figure 50b). Since in the middle of the suit there are four contacts fewer than on the outward sides, two conductors on each connector were cut off. As a next step about 10 mm of insulation of each wire was removed and a 15 mm piece of heat shrinkable tubing was put on each one (Figure 50c). The bare strands of each conductor were bent in the half and hooked around the corresponding bundle of conductive thread (Figure 50d). After a drop of superglue had been applied to fixate the conjunction, the heat shrinkable tubing was slid over the joint and shrunk as described before (Figure 50e). Motion Suit – Construction 88 Gerstweiler, Vonach, 2011 a. b. c. d. e. Figure 50: The Connection to the Ribbon Cables Motion Suit – Construction Gerstweiler, Vonach, 2011 89 7.5. Active Markers Georg Gerstweiler It is not possible to attach the infrared emitters directly on the suit without any device to hold it in place. So there is a need for a mounting system that not only accomplishes that, but also meets special requirements. First of all the LED has to be connected with both poles of the power supply without causing an additional electric resistance. This is a challenge, because if working with common electronic circuits it is possible to connect the individual parts with solder. However in this special application the soldering temperature, which normally is between 200°C and 300°C, would burn and destroy the conductive thread. In addition to that, the mounting solution should be long-lasting and mechanically resistant. It has to be attached on the suit in a way that it does not hinder the user or get out of place. Furthermore the LED should be placed a little elevated, in order to benefit from the high emission angle of the model. The convenient replacement of defective devices has to be considered as well. Figure 51: First LED Assembly The first prototype that has been built to connect the conductive thread with electronic devices was developed for the first head band prototype (chapter 7.1). It is based on a plate which has four electric pads on one side, each of them with two drilled holes (see Figure 51). The two ends of conductive thread from the seams near the marker position are knotted and glued to the two outer holes and the plate itself is then glued on the underlying cloth in order to keep it in place. Two metal pins are soldered on this plate, fixating and at the same time electronically connecting another second board on top of it, which holds the infrared LED. Although this version did fulfill basically all requirements, in Motion Suit – Construction 90 Gerstweiler, Vonach, 2011 the end the effort to produce all sockets would be unreasonably great. In addition the change of an emitter would involve a high risk of burning the surrounding material, because the top plate is only removable through unsoldering. Moreover the construction has sharp edges and is not as small as desired. Due to these facts the authors decided to develop another mounting system, in order to meet the requirements more satisfactory. Figure 52: Components for Marker Assembly Figure 53: Final Marker Assembly Figure 54: Marker Assembly including Foam Craft Patch Figure 52 shows the components and alterations for the final marker mounting concept. It is built up of just three different components, namely one socket, mounted directly onto the suit, another for the LED which can be plugged into the first one and finally the LED itself. The socket only consists of a so-called “socket terminal strip” (see Figure 52 on the top) with four pins, whereat two of them are reshaped by the authors to form an eyelet. The resulting two eyelets can then be carefully connected with the conductive thread through a knot and a drop of super glue (see Figure 53 on the top). Furthermore the whole socket is glued onto the cloth in order to stay in place. Motion Suit – Construction Gerstweiler, Vonach, 2011 91 All four pins of a shortened right angle “pin strip” (Figure 52 on the bottom) are removed and instead the contacts of an infrared LED are put through the two free center holes. The two outer pins are turned around and pushed back with the shorter end into the strip. Then each contact of the diode is soldered onto one of them. The individual steps can be seen in Figure 52 in the middle. The resulting component fits into the previously constructed socked (Figure 53). With this system the LED is located above all other components and is able to emit light over 180°. In addition to that it is possible to change a broken LED very quickly if a few spare markers are constructed in advance for replacement. It also provides the opportunity to attach emitters with different wave lengths for example in order to use color coding on the motion suit. Because the element with the light source is only plugged into the socket, there is a risk of losing the component due to fast movements. However the emission regulation as described in the following (see also chapter 6.1.6) can be used to fixate this part sufficiently. Since the suit consists of diverse fabric materials with slightly varying colors, they reflect the infrared light in different ways. To achieve a similar marker appearance in the camera images, the authors decided to surround every LED with a circular patch (see Figure 54) with a diameter of about 3.5 cm made of foam craft sheets. This patch is able to absorb the infrared light and therefore prevents unwanted distortions or reflections, which could wrongly be detected as “ghost markers”. In the center of the patch there is a small cut-out through which makes it possible to plug the piece containing the illuminant into the socket (see Figure 54) and at the same time holds the LED in place. The patch itself is attached to the suit with special fabric glue that is still elastic even after it has dried. 7.6. Control Box Georg Gerstweiler The central circuitry of the motion suit is located in a plastic control box and combines all parts required to light a desired marker layout at the intended time. Chapter 6.2 already described the individual components (also visualized in Figure 55) that are needed to perform these tasks. First tests also showed that Motion Suit – Construction 92 Gerstweiler, Vonach, 2011 the assembly of these parts is fully functional and meets the requirements. Now the chapter at hand deals with the actual construction of the control box with the focus on the combination and interrelation of the components. There is a the high number of wires on the whole motion suit, which lead to the electronic components in the control box and therefore require a quite complex routing on the inside. As a result of this a custom printed circuit board layout was designed (see Figure 56). Because every marker has individual conductive paths, a total of 120 wires entering the control box have to be routed correctly to 19 pins of the MAX6954 LED driver. This is accomplished through a matrix circuit on the PCB. To design the board the software EAGLE introduced in appendix B.3 was used. All paths were placed on two layers, namely the top and the bottom of the board. The size of the board is constricted by the size of the connectors. Figure 58 at the end of the chapter shows the schematic layout of the printed circuit board and the connections between the components. The final PCB measures 7 cm by 10 cm. Figure 55: Individual Components of the Control Box Figure 56: PCB Layout Due to the characteristics of the circuitry it is possible to use a lithium-ion polymer accumulator with 7.4 V to be mobile or an external power supply for longer training sessions. The minimum potential difference at which the control box can work reliably is defined by the specification of the Arduino Nano with at least 7 V. The Arduino is directly connected to the power supply, because its recommended input voltage is between 7 V and 12 V. The LED driver is powered over an individual connection to the power source. It needs two Motion Suit – Construction Gerstweiler, Vonach, 2011 93 capacitors between the power lines very close to it, in order to balance voltage fluctuations. Both devices, the Arduino Nano and the MAX6954, have to share a common ground, since an SPI connection exists between them. The MAX6954 is powered with 5 V to overcome the high resistances caused by the conductive thread. The feasibility tests showed that the LED driver is able to deal with 67.5 Ohm maximum when connected to this potential difference. It is not possible to use the built-in voltage-limiter of the Arduino Nano to power the LED driver, because it is only constructed to withstand a maximum of 500 mA with 5 V. Therefore an additional voltage-limiter has to be used. The LED driver itself absorbs a possible voltage overhead that is not dissipated by the LEDs or the resistance of the conductive thread. In order to relieve the driver the authors decided to fit particularly those connections that lead to markers located close to the control box with additional resistors, in order to keep the resistance at an even level of about 50 Ohm. For a more convenient handling, the authors added a number of features, described in the following. A green status LED on top of the box connected to the Arduino Nano allows giving feedback to the user. If the control box is connected to a workstation that is running the “Motion Suit Manager” (see chapter 7.7.2) which allows the configuration of the marker layout, this LED can for example indicate an error during a data transfer. It also shows if the firmware has finished starting up and the current marker layout has been loaded successfully. While using the motion suit with a mobile power supply, it is useful to see whether the battery is low. Therefore a so called LiPo saver is connected parallel to the power lines. It signals the user through a red LED that the battery should be charged or changed. Furthermore the whole suit can be switched on and off with a toggle switch located on the side of the box. The size of the control box is limited by the dimensions of the designed printed circuit board and the size of the accumulator. Figure 57 shows the readily assembled PCB already built in a tight fitting plastic box. Because the components in this box offer more functionality than required at the moment, a connector with 34 poles is provided to attach additional devices without the need for producing a new control box. Furthermore there are two adjustable Motion Suit – Construction 94 Gerstweiler, Vonach, 2011 jumpers added to the board. They can be used to reroute the wiring of the SPI connections between the Arduino Nano and the LED driver in order to address additional SPI devices plugged to the upgrade connector. Figure 57: Finally Assembled Control Box Motion Suit – Construction Gerstweiler, Vonach, 2011 95 Figure 58: Control Box Schematic 7.7. Software for Operation and Management Emanuel Vonach In this chapter the software is presented which was implemented by the authors in order to open up the whole range of functions of the motion suit. First the Motion Suit – Construction 96 Gerstweiler, Vonach, 2011 firmware for the Arduino Nano is described, which is not only responsible for the activation of the desired marker patterns on the motion suit, but also handles tasks like the communication with an operating station or the triggering of certain patterns per I²C. Subsequently a simple interface protocol is described, which was implemented to allow a reliable transmission of commands and the exchange of information over USB. Finally the authors introduce another application implemented in this context: the “Motion Suit Manager”. This piece of software offers an illustrative graphical user interface and allows creation or adaptation of marker patterns and their transmission to the control box of the motion suit in a comfortable manner. 7.7.1. Firmware Emanuel Vonach As described in chapter 6.2.2 the authors decided for an Arduino Nano as central control unit for the motion suit. However in order to cover all the required operation tasks it had to be fitted with an elaborate firmware, which was implemented with the Arduino IDE [Arduino '11] (appendix B.2). Amongst other functions this program has to be able to handle instructions received from an operating station over the USB interface, for example to activate certain markers, as well as to manage the LED matrix driver via SPI. The information about which markers are to be active at a particular time is written in a lookup table in the static memory of the Arduino (SRAM). Several different patterns can be stored as “frames” so that they can be triggered at the moment of choice. The lookup table is structured as a three-dimensional matrix with the indices “frame number”, “digit” and “active segments”. The digit can be seen as the row of an LED matrix circuit, while the active segments describe the columns as a 2-byte binary pattern. These 16 bits directly correspond to the 16 elements of one row. The lookup table can be managed via USB and is additionally mirrored in the permanent memory of the Arduino Nano (EEPROM) together with the global intensity of the LEDs. In that way the data can be restored if the power source is disconnected or the board is reset, which would in both cases clear the static memory. Motion Suit – Construction Gerstweiler, Vonach, 2011 97 For fast external frame triggering, for example over an RF module, an I²C interface is provided. The I²C implementation as well as the EEPROM access is an adaptation from the Wire.h respectively EEPROM.h code provided with the Arduino library [Arduino '11]. For operation without a trigger as well as after startup or reset, the first frame in the lookup table is loaded as default. Although the EEPROM is able to store up to 64 frames (8 bytes each, which sums up to 512 bytes), the authors decided to limit the maximum allowed number of frames to 20. The authors also implemented a simple method for feedback and error handling at runtime: The provided status LED of the control box is used to signal different messages by various pre-defined blinking sequences, depending on the intended feedback. For example one blink represents a simple acknowledge signal, three blinks indicate that an error occurred during SPI communication, and so on. Additionally the specific error code is sent over the USB interface, if connected. To elaborate on the features of the firmware, in the following the routine of the program is presented: First after startup the lookup table is retrieved from EEPROM and loaded into SRAM. The device joins the I²C bus by assuming an available address. Since the communication between the Arduino Nano and the MAX6954 matrix driver takes place over SPI, the corresponding control register has to be initialized. This register contains information about the properties of the interface: It defines that the most significant bit (MSB) is transmitted first, the data has to be sampled at the leading edge of a clock pulse, the clock signal is low when the bus is idle and no interrupts are used. It also declares the Arduino Nano as master device and sets the clock rate to maximum (system clock speed divided by four, which is the fastest possible setting of 4 MHz). The communication procedure itself is described in chapter 6.3. After the communication between the Arduino board and the matrix driver has been established, the driver’s control registers are set. In that way it is configured to operate four rows with up to 16 LEDs each. As already mentioned the authors decided for employing all available columns rather than using more rows, in order to reduce the pauses between the active cycles (see chapter 7.1). If even longer active cycles are desired and fewer markers are to be used, Motion Suit – Construction 98 Gerstweiler, Vonach, 2011 for example only the torso of the suit, the number of active rows could be reduced even further. In addition the decode mode and the test mode of the driver are disabled and the global intensity for the LEDs is set according to the information stored in EEPROM. After that the default frame is loaded and the readiness of the system is signaled over a blink of the status LED. As long as no I²C activity is detected, the default frame stays active. However as soon as a transmission containing a valid frame number is received over this interface, the system switches to triggered operation and activates the corresponding frame. If the operating station is connected, an acknowledge message is sent over the USB port as well. The system is also continuously listening for activity on that bus, since the lookup table and the intensity setting is managed over this communication channel. For that purpose the authors developed a simple 4-byte protocol: In order to set an LED in a particular frame active, the reference number of that frame followed by the LED identification has to be sent. The command is finalized with a semicolon. The firmware parses the marker ID and adjusts the specified frame in the lookup table as well as the backup in permanent memory. For example the message “12H.1;” sets the first marker on the head in frame 12 active. If the first byte received contains a semicolon, this indicates a special command. If the semicolon is followed by the character “C” and a frame number (e.g.: “;C12”), this instructs the system to clear the specified frame in the lookup table as well as in the permanent memory. The character “A” instead of a specific frame ID (e.g.: “;CA”) empties the whole lookup table, again in SRAM as well as in EEPROM. Likewise an “R” together with a number (e.g.: “;R12”) orders the Arduino to read a specified frame from permanent memory and to transmit the currently stored pattern to the operating station. Other commands are assigned to the letter “I” followed by a value (e.g.: “;I7”) to set the global intensity and the message “;T” which enables or disables a test mode. To make sure that the transmissions are received, every command mentioned up to here is confirmed by the Arduino with an acknowledge message. In case something went wrong it transmits the corresponding error code. In order to test the Motion Suit – Construction Gerstweiler, Vonach, 2011 99 connection the message “;P” acts as simple “ping” command which lets the system generate an instant acknowledgement. Although a similar protocol could have been realized far more efficient, the authors decided to keep it at least close to human readable since speed was not a priority in this case. In that way the motion suit can also be managed manually over any serial command interface without having to calculate which bit-pattern corresponds to which command. 7.7.2. Motion Suit Manager Georg Gerstweiler One of the important requirements for the motion suit is the individual control of all markers. The appropriate hardware has already been chosen for the control box to enable this feature and the firmware is also ready to accept commands to switch between marker layouts. This chapter describes an application called “Motion Suit Manager” that was designed to communicate with the motion suit. It allows the user to manipulate the marker settings over a graphical user interface and was implemented with the open source programming language Processing [Processing '11](appendix B.1). The circuitry in the control box is able to store multiple marker layouts, also referred to as “frames”. Since the permanent memory of the ATmega168 is limited, the authors decided to allow 20 different setups. In order to update this data, a USB connection has to be established between the control box and a workstation on which the Motion Suit Manager is running. The possibility to predefine multiple marker layouts was implemented to prepare for the planned upgrade of the control box with a wireless connection, which would enable the usage of a trigger signal to switch the marker layout during a training session. Though in this project changing the layout can only be performed a priori with a wired connection to a PC and the motion suit always starts up with the first frame loaded. The interface (see Figure 59) shows a front view and a back view of the motion suit, displaying all available markers. Each marker position is connected to a button, which disables or enables the LED in the currently active frame. First all settings are stored and displayed only locally on the PC. In order to upload one or all frames, it is necessary to have an established serial Motion Suit – Construction 100 Gerstweiler, Vonach, 2011 connection to the control box. The Arduino Nano, which is responsible for receiving the data, is programmed to restart each time a serial connection is established or closed. For that reason an upload takes quite some time, since it has to be waited until the startup is completed. The application is sending the data over the USB connection with a reliable protocol the authors developed for that purpose which is described in previous chapter. In addition it is also possible to set the intensity control register on the MAX6954 LED driver to change the global intensity of the markers respectively the display duty cycle. In order to achieve different intensities without changing the resistor connected to the ISET pin of the driver, the pulse-width modulator scales the segment current of 40 mA in 16 steps. For that reason 16 intensity steps can be chosen in the user interface and uploaded individually for each frame. Figure 59: Motion Suit Manager – User Interface for Marker Layouts Motion Suit – Construction Gerstweiler, Vonach, 2011 101 7.8. Costs Georg Gerstweiler After all components have been assembled to a fully functional active motion suit, in this chapter the incurred expenses for the actually employed materials are presented, whereas all named prices are including VAT. It must be pointed out, that it was not always possible to distinguish between the materials used for test purposes and those actually applied on the motion suit, since some were partially used for both. For that reason the cost report in the following is only an approximate calculation. This overview also illustrates how many different materials were actually bought in order to build the motion input device described above. Table 1 shows a list of materials, which have a high contribution to the final costs, either due to a large quantity like in case of the infrared LEDs, or a high price per piece like the Arduino Nano. These eight items are resulting in a total of 356 EUR. # Product Price (EUR) / Piece 1x Printed Circuit Board Production 70.00 1x Base Material: Trousers, Shirt, Headband 55.00 2x Insulation Material: Textiltransfer-Gel 20.00 1x Arduino Nano 40.00 5x Conductive Thread (200 ft per coil) 20.00 15m Heat Shrinkable Tubing 2.00 70x HSDL-4400 IR-LEDs 0.30 Table 1: Costs of Most Significant Items The construction process also required many less significant materials. These are listed in Table 2. Although not every minor piece is itemized separately, nevertheless they are also included in the sum. Even if one item alone does not cost much, all together sum up to a considerable price of 205 EUR. Motion Suit – Construction 102 Gerstweiler, Vonach, 2011 Product Price (EUR) Sewing Accessories: Hook and Loop Tape, Elastic Webbing, Threads, ... 90.00 Various Consumables: FLUX, Super Glue, Fabric Glue, Solder, Silver Conductive Paint 40.00 LED Mounting: Socket Terminal Strip, Pin Strip 30.00 Connectors 25.00 Control Box Components: LiPo Saver, Resistors, Capacitors, Box, ... 20.00 Emission Regulation: Black Foam Craft Sheets 3.00 Table 2: Costs of Less Significant Items Considering the whole equipment acquisition for the active motion capturing suit, about 560 EUR were spent. In addition to that about 200 EUR were necessary in order to test materials for their suitability. If a ready-to-sale product is desired, a further reduction of these cost would be possible, since some of the hardware would be much cheaper when ordering multiple pieces, especially the printed circuit board. Furthermore the development platform Arduino Nano could be replaced through an individual ATMEL chip, which would cost only a fraction. The overall costs of the developed prototype in this work are comparable to conventional passive suits available on the market, which start without markers or any functionality at about 200 EUR like the basic suit of the company “3x3 designs”[3x3 '11]. Evaluation of the Motion Suit Gerstweiler, Vonach, 2011 103 10. Evaluation of the Motion Suit Emanuel Vonach In order to make a basic evaluation of the project presented in the course of this work, a series of user tests was performed. These tests did not aim on a comparison of different approaches, for example in the view of performance or efficiency, but on the demonstration of the general suitability for educational purposes in the area of motion skills training and to identify potential opportunities for improvements. First the test setup is explained in detail, followed by a summary of the observations made during the actual tests. Finally the results and the inferred conclusions are presented. 10.1. Setup of User Tests Georg Gerstweiler Before the actual user tests can be performed, detailed preparations are required. In order to evaluate the developed system only qualitative methods are used including a semi-structured interview and a direct observation. Before going deeper into the details of the planned test protocol, the employed hardware and software for the setup is described. For the tests an iotracker setup with eight cameras is employed. They are mounted on two opposite walls covering a tracking volume of about four by four meters and three meters in height. Furthermore the algorithm presented in chapter 4.4 is applied to detect all joints of the tracked skeleton and generates a well structured extensible markup language file, which can be retrieved over a network link. In addition to that a visual output is required to see the actual performance instantaneously on a screen, while running a motion tracking session. Therefore an educational virtual reality application called “MoCapGym” is used, which was developed by the authors in another project. This software receives the skeleton data over a network link and is not only able to map this information to a virtual avatar in real time, but can also record a whole capturing session. MoCapGym animates two avatars, one with the data of a previously recorded session (teacher avatar) and another one (student avatar), which Evaluation of the Motion Suit 104 Gerstweiler, Vonach, 2011 corresponds to the real time data received from the tracking system. During a capturing session the application can compare both avatar poses and visualize the difference of the orientation in every joint with the help of a color mapping, starting with green, if the joint angles are identical and going smoothly towards red, if the current joint angle differs. The user is thereby encouraged to imitate the movements of the teacher avatar in order to get a good comparison rate. During the user tests a projector will show both avatars in front of the user on the wall, behaving like a virtual mirror. Figure 60 presents a screenshot of the visual output during a training session. The motion suit itself is powered by a mobile voltage source during the user tests. The marker configuration is set to allow a tracking of the following joints: knees, hips, pelvis, neck, shoulders and elbows. For the comparison it is important that the recorded data includes the same joints, since every additional or missing one cannot be rated. Figure 60: Screenshot of MoCapGym during a training session in comparison mode. For evaluation of the suit, the method of direct observation and a semi-structured interview are applied. This requires additional preparation before the tests can actually be performed. It is almost impossible to observe and note every detail during the tests. For that reason on the one hand the test person is captured in video and audio and on the other hand the visual output of MoCapGym is recorded over the whole time. Evaluation of the Motion Suit Gerstweiler, Vonach, 2011 105 The first interview focuses on collecting several facts and experiences of the participant in a short questionnaire, presented at the end of this chapter. The first part is filled out by the user himself and concerns mainly general facts. The second part however concentrates on the experiences during the test and is used as a guideline in a qualitative semi-structured interview. This approach makes it possible to get more detailed and personalized feedback and enables the supervisors to react to answers and ask additional questions for a better understanding. The user tests are conducted according to a well structured protocol prepared by the authors. Although there is no time limit planned, the estimated duration of one session is about one hour, depending on the motivation and interest of the test subject. At the beginning every participant gets a short explanation about the procedure and is informed that not he, but the developed system is going to be examined. After the first part of the questionnaire has been completed, the user is introduced to the subject of motion tracking and the hardware that is applied in this context. In the next step the test person is asked to put on the motion suit. If it is possible for him to accomplish this independently there will be no help given by the test supervisors. At this point all recording devices and the tracking system are activated. The algorithm for motion tracking requires a short calibration procedure (see chapter 4.4) wherein the participant has to move all joints of interest, if possible simultaneously, in a short time period. In order to demonstrate the test person a recommendable chain of movements for a good calibration result, the software MoCapGym is started and a previously recorded example session is presented to the user through the teacher avatar. After a successful calibration the main tracking session begins. At this stage the supervisors have to watch carefully every movement of the test person and the visual output at the same time, in order to be able to analyze any unexpected behavior of the user, the motion suit or the tracking system. The test person now sees successively four different motion sequences several times. Due to the visual comparison of both avatars the participant is urged to imitate the course of motions of the teacher avatar, in order to get as much green parts on the student avatar as possible. Evaluation of the Motion Suit 106 Gerstweiler, Vonach, 2011 After all levels have been completed everyone is given the possibility to move around freely and experiment with the system. In this way the supervisors can observe the performance of the suit with some more natural or perhaps also unexpected movements. After this all recording devices are stopped and the user is asked to take off the motion suit. Finally the qualitative interview is conducted with the test person, guided by the second part of the questionnaire. With this interview the supervisors try to explore the personal experiences of the test person throughout the whole session. Evaluation of the Motion Suit Gerstweiler, Vonach, 2011 107   Development of an Active Motion Capture Suit for Teaching Motion  Skills      Interview for evaluating the obtained research results.  Fragebogen zur Einschätzung der erzielten Forschungsergebnisse  General facts Part 1/2:  Allgemeine Information Teil 1/2:      Age/Alter:    _____________    Body height/Größe :   _________________    Sex / Geschlecht:    FEMALE / WEIBLICH     |   MALE / MÄNNLICH     Do you already have any experience with tracking  systems?    Haben Sie bereits Erfahrung mit Tracking Systemen?      YES / JA   |  NO / NEIN      If “Yes”, please elaborate / Wenn ja welche:    _____________________________________    _____________________________________    How would you describe your attitude towards this user  test?    Mit welcher Einstellung gehen Sie in den Versuch?      WEARY / MÜDE      EXCITED / GESPANNT    BORED / GELANGWEILT    RELAXED / ENTSPANNT      and / und: _____________________________    ______________________________________      To which area of expertise would you assign yourself?     Welchem beruflichen Bereich würden Sie sich  zuordnen?          Do you agree to be captured by audio and video during the  test?    Sind Sie mit einer Audio und Videoaufzeichnung während des  Testes einverstanden?  YES/JA    |  NO/NEIN       Date:      ___ ___ ___ Test no.:  _________ Begin:  ____ : ____ End:        ____ : ____ Evaluation of the Motion Suit 108 Gerstweiler, Vonach, 2011   Personal experience Part 2/2:  Persönlicher Eindruck Teil 2/2:      Please describe your impressions while putting on the motion suit!    Bitte beschreiben Sie kurz Ihre Eindrücke beim Anziehen des Anzuges!                  How would you rate your freedom of motion  during the tracking session?    Wie frei konnten Sie sich während der Übung  bewegen?      GOOD/GUT 1  | 2  | 3  | 4  | 5  POOR/SCHLECHT    Did you notice something hindering during the tracking session?    Ist Ihnen während der Übung etwas Störendes aufgefallen?                       Which grade would you give the motion suit?    Welche Schulnote würden sie dem Anzug geben?        GOOD/GUT 1  | 2  | 3  | 4  | 5  POOR/SCHLECHT    Why / Warum?    _________________________________________    _________________________________________    _________________________________________      THANK YOU FOR YOUR TIME!  VIELEN DANK FÜR IHRE ZEIT!  Evaluation of the Motion Suit Gerstweiler, Vonach, 2011 109 10.2. User Test Observations Emanuel Vonach After the careful preparations described above, in the chapter at hand the observations made during the actual tests are summarized. Since qualitative test methods are used, only five persons were selected as participants, one female and four male. All of them were between 25 and 30 years old and between 175 and 186 cm tall. During the preliminary questionnaire all of them claimed that they already had to do with tracking systems, though aside from eye tracking in one case, none of the participants ever actually used one. Regarding their current condition all stated to be relaxed and motivated, only two were tired. Nobody declared himself stressed or bored. When putting on the motion suit, all participants did it basically right. Although all of them seemed a little confused by the conductive thread bridges on the inside of the suit, they were able to do it intuitively correctly and mostly without help. Only the technique to apply the straps for the markers on the feet required a brief explanation by the supervisors. Notably all participants handled the suit with extreme care, obviously because they worried about damaging something. During the qualitative interview conducted later, most test subjects did not state any difficulties in this context though. Only two of the taller persons described the suit as a little too tight and one additionally complained that the trousers were too short and putting them on was complicated for him. Then again one person specifically pointed out that the head band was very comfortable. The calibration of the tracking system ran smoothly. Although a functional motion sequence was demonstrated by the supervisors, the execution through the participants was unique regarding speed, precision and balance every time. Nevertheless it was possible to infer a skeleton for every user without problems. When the participants subsequently were allowed to experiment a while with their avatar to get to know the system, it was noteworthy that at first all test subjects only moved very slowly and carefully and observed every reaction in the projection on the wall. After a while their movements got visibly faster and more self confident. During the actual training phase that was part of the test, the tracking proved to be pretty stable. In particular head, arm and upper body movements worked Evaluation of the Motion Suit 110 Gerstweiler, Vonach, 2011 very well. Even rather critical situations for optical tracking systems like bending forward or sitting down were possible to a reasonable extent, although one person stated in the interview later, that he was slightly constricted by the bridges inside the suit when leaning forward. The authors themselves observed inaccuracies when the test subjects bent their knees or elbows, in particular with smaller or especially lean persons. During one test session they applied an additional tape at the upper arm to successfully counter this issue. Regarding the training itself (Figure 61), every participant was able to successfully learn all the presented movement sequences and reproduce them later. Everybody was enjoying the training sessions and was motivated by the challenge to get as much green areas on his avatar as possible. Without being asked to, notably all of them played and experimented further with the system even after the actual training session was over. In particular one participant who assigned himself to the area of e-learning was very enthusiastic about the possibilities for applications of the motion suit. When the test subjects were asked in the concluding questions to give grades regarding their freedom of motion during the tracking session, three of them checked “2” and two “1”. As overall assessment for the motion suit three persons graded it with “1” and two with “2”. Noticeably the latter were given by the taller persons though. Altogether each test lasted about 1.5 to 2 hours and was conducted in the afternoon. The actual training session took about 30 minutes. Figure 61: Participant During Training Session Evaluation of the Motion Suit Gerstweiler, Vonach, 2011 111 10.3. Results Emanuel Vonach The observations from the user tests provide considerable information about the success of this project. Putting on and off the suit has worked very well, which supports the authors’ decision against a full body solution based on straps. Probably the straps for the feet should also better be replaced by another concept, although after a short explanation and with minimal help all the test users were able to fixate them correctly. An alternative worth considering could for example be markers on flexible wires which could be affixed directly on the user’s shoes. In any case they should be removable, in the same way as the headband. Furthermore the noticeable carefulness of the test subjects while putting on the suit suggests that a more robust track suit as base material might be worth considering, even though the selected model is not as fragile as it may seem and there was actually no damage detectable after all tests. However especially taller persons did not dare to pull hard to get the suit on despite the fact that it was quite tight for them. In this context it also seems to be a good idea to provide the motion suit in different sizes, which is also supported by the observed tracking imprecision when bending the knee or elbow. These problems are very likely caused by markers that move too much in relation to the tracked limb. A better fitting suit as base material, additional hook and loop tapes mounted on critical locations or a more tolerant tracking system would be possible solutions for the issue. Nevertheless since muscles and skin always have an influence as well, the markers can never be fixated absolutely static anyway. The users’ assessment regarding the freedom of motion is also quite promising, because the only complaints seemed to arise more out of the overly care about the material than of an actual impediment. Although the authors had been concerned that the control box could get in the way, this does not seem to be the case at all, since none of the test subjects mentioned it. However in the authors’ opinion it might still be a good idea to design an even smaller and lighter box. For example the accumulator used at the time would allow more than two hours of continuous tracking. Perhaps a model with smaller power Evaluation of the Motion Suit 112 Gerstweiler, Vonach, 2011 capacities would suffice and at the same time help to reduce weight and size of the control box. Concerning update rate and latency, the employed tracking system proved to be adequate considering the fact that after only a short time of familiarization, all users behaved naturally and confidently. Although the system is actually designed to be used with passive markers the constructed active motion suit worked without difficulties. However this fact allowed the authors to include an object with passive markers to be tracked too and thereby successfully combine active and passive optical tracking. In order to accommodate the employed skeleton estimation algorithm (see chapter 4.4), the LEDs on the joints of the tracked person were deactivated in the marker layout. This demonstrates that the individual control of the markers works reliably and the layout can be adapted for any use case as far as the positions of the mounted LEDs support it. Once all user tests were concluded, the authors removed the active markers as well as the control box and washed the suit manually and very carefully with warm water. After thorough drying they refitted it again with the removed components and it was still fully operational, which proves that the hygienic requirements are met as well. The high level of motivation the participants showed during the user tests allows very promising conclusions in the view of the suitability and applicability of the system for education in general as well as specifically in the area of motion skills and rehabilitation. Since especially autonomy and motivation are crucial factors for successful learning [Goudas '95], the employment of virtual reality environments proves to be reasonable and can be realized in practice. The authors assess these observations together with those described above as corroboration of the central hypothesis and approval that the purpose of this work was achieved. Conclusion and Future Work Gerstweiler, Vonach, 2011 113 11. Conclusion and Future Work Georg Gerstweiler This work presented the development and analysis of a human motion input device, which was especially designed for training motion skills. Therefore it was necessary to combine the topic of motion tracking with various other fields of expertise including microcontroller programming, fundamental electronics, wearable computing and last but not least sewing skills. The combination of all of these very differing areas posed a very special challenge and the authors had to go through multiple familiarization phases. Because of these circumstances the overall development period took about than one and a half year. Figure 62: The developed active motion suit in combination with a passively tracked head mounted display. The resulting motion suit (see Figure 62) was successfully tested in several tracking sessions. Although the overall concept and developed devices proved to be adequate, the authors think that some minor details of the motion suit could be improved or should at least be kept in mind if this work is pursued further. For example it would be better to provide the motion suit in various Conclusion and Future Work 114 Gerstweiler, Vonach, 2011 sizes, since it was not possible to adapt the base material to accommodate for people with extremely different body dimensions. Considering this, a more flexible way of positioning the markers would also be desirable, because the body size has a big influence on their actual location, in particular if they are intended to be near the joints. Also a higher number of markers could be beneficial. Apart from these minor issues, especially the conductive thread performed very well in this application, although it was necessary to sew a total of over 900 meters to connect the control box with the individual LED markers. This involved quite a big effort. A machine-made production would be much more convenient and also save a lot of time. The major advantage of using so much thread was a relatively low resistance and an almost indestructible wiring. Another central device proved to be an optimal choice, namely the matrix driver. It made many special features possible and at the same time kept the power consumption of the LEDs low. The designed prototype was not intended to be a ready-to-sale product, but in the authors’ opinion it would be possible to achieve a well usable product for end users if some minor concepts were adapted. These adaptations mainly concern the base material and an even more compact design of the control box. Furthermore the material costs of about 560 EUR including VAT can be further reduced and would be competitive on the market. Since the motion suit has a very broad range of functionality and various features, further research of their performance during tracking sessions in a variety of applications would be reasonable. The LED mounting system for example supports the usage of all kinds of LEDs and could therefore be used for color coding of different parts of the body. In addition to that the already prepared possibility of triggering different marker setups would be a very interesting research area if synchronization with the tracking system can be established. Finally intensive research about the learning progress with a motion skill training system compared to traditional methods seems to be a valuable future work. Appendix Gerstweiler, Vonach, 2011 115 Appendix A. Electrical Fundamentals Emanuel Vonach In order to follow the considerations and decisions in this work a basic understanding of electronics is required. For that reason electronic fundamentals are described subsequently in this chapter. First basic electrical properties and their interrelations are presented. Afterwards elementary electric circuits are illustrated. Unless stated otherwise, the information presented in this chapters and its subchapters are based on the book “Basic Electronics” by B. Grob [Grob '99]. A.1. Voltage (V) Georg Gerstweiler Voltage is described with the unit “volt”, which is named after Alessandro Volta. In order to generate voltage it is necessary to do physical work by separating electrons from protons. These two unlike charges establish a potential difference, which describes the ability of doing physical work, either by attraction or by repulsion. Under natural circumstances the stress between these two energies draws the lighter electrons (1/1840 of a proton) towards the protons to establish a neutral condition. In other words voltage could be described as the work that has to be done in order to produce a balanced system. In order to measure voltage there are always two contacts needed, connected to two different amounts of charges. If there is no potential difference there is also no voltage. Figure 63 shows three examples with the same potential differences but different charges. Figure 63: Potential Difference Appendix 116 Gerstweiler, Vonach, 2011 To describe voltage in numbers the unit “coulomb” plays an important part. Charles A. Coulomb defined a charge of 6.25 E1018 electrons to be one coulomb (C). So the potential difference of one volt exists if one “joule” of work has to be done in order to move one coulomb of electrons. Joule (J) is a unit of work or energy: 1 1 1 In general the voltage specifies the amount of current which can be used. The bigger the potential difference the more current can flow. A.2. Current (A) Georg Gerstweiler “Current” describes a flow of charge between two poles connected to a potential difference. Materials like copper do have free electrons which can be easily forced to move through the material by applying a potential difference. In every ordinary application the current exists instantaneously all over the wire and has the same value at every point. Figure 64 shows two charges, a positive one +Q and negative one -Q, at the two ends of a wire. The electron “E”, at the left side of the wire, has moved from the negative charge towards the positive one into the next atom. For that reason this atom can free one of its own electrons, again towards the positive end. This goes on until the last atom frees an electron (electron 7 in Figure 64) into the positive charge of the voltage source. Now a continuous flow of electrons exists whereas the more electrons are moving through the wire the higher is the current. Since current is about intensity the unit for current is “I”. 1E 2 3 4 5 6 7 + Q- Q free electrons in wire Figure 64: Electric Conductor Appendix Gerstweiler, Vonach, 2011 117 In order to measure current the value “ampere” was established by André M. Ampère. One ampere is measured, if one coulomb of electrons is moving through a given point in one second. Thus there can be a potential difference without current but never a current without a potential difference. A.3. Resistance (R) Georg Gerstweiler An uncontrolled flow of current creates a lot of heat and must therefore be regulated by a “resistance”. Copper for example has a very low resistance because it has a large number of free electrons which can be moved very easily by an applied voltage source. In contrast to copper, carbon has a small number of electrons so there is more potential difference needed to create the same amount of current. The official symbol for resistance is “R” and it is measured in “ohm”. For the abbreviation the Greek letter “Ω” is used. A resistance of one ohm has to create 0.24 calorie of heat as soon as one ampere flows through it. a. b. c. Figure 65: Resistor Symbols: a. DIN, b. ANSI, c. Resistor Color Coding In almost every electronic circuit a resistance is needed to regulate a flow of current. The electronic component which is used for that purpose is called a resistor. Figure 65a and b show the electronic symbol that is used in a circuit diagram. When choosing a resistor there are three values which describe the device: resistance (R), tolerance and power rating (W). The first value R has to be exactly dimensioned. Depending on the application it can differ between one ohm and several megaohms. To recognize the value of a resistor every single device has color-coding stripes directly on the component (Figure 65c and Figure 66a). On smaller models, so called “surface mount resistors”, the value is directly written on it (Figure 66b). It is possible to combine different resistors in series to get the needed value. Furthermore no resistor is completely Appendix 118 Gerstweiler, Vonach, 2011 accurate. Depending on the tolerance resistors do have inaccurateness in the range of ±0.1 % to ±5 %. The last value which should be paid attention to when choosing a resistor is “watt” explained in the following chapter. In this context it describes how much volt respectively current the resistor can withstand, without generating so much heat that the device would be destroyed. Figure 66a - c shows the most common types of resistors. a. b. c. Figure 66: Types of Resistors: a. Metal Film Resistor, b. Surface Mount, c. Variable Resistor – Potentiometer A.4. Electric Power (P) Georg Gerstweiler The “electric power” (P) is measured in “watt”, which is named after James Watt. It describes the work that is done by one volt in one second by moving one coulomb of electrons. So the power that is dissipated in a system can be calculated with: ; ² ; This formula is based on a fundamental rule of electricity, the “Ohm’s Law”, which is described in the following chapter. A.5. Ohm’s Law Georg Gerstweiler “Ohm’s Law” is a basic instrument when dealing with electronic circuits in a closed path and was found by Georg Simon Ohm around 1826. It describes the Appendix Gerstweiler, Vonach, 2011 119 linear correlation between the three main values in electronic systems, which are voltage, current and resistance. If there is a circuit with a resistor and a potential difference like it is shown in Figure 67, a current is flowing through the resistor R. Figure 67: Simple Electronic Circuit With the help of Ohm’s Law it is possible to calculate one missing value. There are three ways to use this formula. If there is a potential difference U connected to a resistor R a specific amount of current I is flowing though the resistor. U R If there is a specific potential difference and current needed, this formula calculates the resistance that allows this current to flow. U I If there should flow a specific current through a given resistor, the potential difference therefore can be calculated. ∗ Furthermore Ohm’s Law shows that it is possible to get 40 mA with 1000 V as well as with 1 V by just changing the value of the resistor. In many situations a specific amount of current is needed, because of the requirements of a particular consumer like an LED. By using a battery or any other voltage source Appendix 120 Gerstweiler, Vonach, 2011 the potential difference is normally not immediately the required one. So a very common situation is to choose the right resistor for a circuit. A.6. Capacitor Georg Gerstweiler Another frequently used element in an electrical circuit is the “capacitor”. This device is able to hold a limited electronic charge. Its assembly is quite simple. It has two conducting plates that are separated by an insulator (see Figure 68). To charge the device it is necessary to connect these two conductors with a potential difference like a battery. This causes the capacitor to attract electrons on the one side and repel them on the other side, until the potential difference between the two conductors equals the value of the applied voltage source. Figure 68: Structure of Capacitors Different capacitors vary in the amount of charge they can hold. The higher the capacitance (C) the more charge can be stored, depending on the size of the surface of the conducting plates. The potential difference only depends on the applied voltage source. The capacity is measured in farad (F). Typical capacitors have a capacitance in the range of picofarad to microfarad. One farad enables the capacitor to store one coulomb of electrons, if the applied potential difference equals one Volt. A loaded capacitor can hold the charge with or without a connected power source as long as no path is provided between the two conductors, because of the dielectric stress built up by the electric field of the positive and negative charge. So the capacitor can be taken out of the circuit and the charge still Appendix Gerstweiler, Vonach, 2011 121 remains in the device. If a path is created between the two plates, the capacitor discharges and can therefore be used as a voltage source. When discharging a capacitor the charge drops down to zero or at least to the level of the applied voltage source. This component is nowadays integrated in almost every electronic device and has a very broad application area. With a capacitor it is possible to build both analog and digital information storages like DROMs, EPROMs or flash memory, where the charge is used to store a binary bit. Furthermore it is also employed as energy source over a short period of time, for example to bypass a loss of power when changing a battery. It is also commonly used to deliver high energy bursts in a very short time, to power for instance a flash unit. Because the capacitor is able to charge and discharge in almost no time, it can also be applied as a converter. In complex digital circuits it is commonly used to stabilize the supply voltage to avoid the disruption of any sensitive device like an integrated circuit (IC). A.7. Series and Parallel Circuits Georg Gerstweiler When more components are used in a circuit with a voltage source there are three main possibilities how these components can be arranged. If there is just one path through the whole circuit it is called a series circuit. A parallel circuit on the other hand is formed, if at least two components are connected separately across one voltage supply. The third possibility results out of a need to mix these two types of circuits together in order to be more flexible. Each of these types has its own individual behavior, whereas the principle can be applied to AC and DC circuits. A.7.1. Series Circuit Georg Gerstweiler To create a series circuit it is necessary to connect just the ends of each component together in a successive way to get one closed path circuit. The current now takes its way from one side of the voltage source through one or Appendix 122 Gerstweiler, Vonach, 2011 more devices to the other side. The order of the components such as resistors does not have any effect on the amount of current flowing through. This circuit must not have a crossover with another path where the electrons could find another way to get back to the voltage source (see Figure 69). A main characteristic of series circuits is that the current is the same all over the circuit, because there is just one path. Figure 69: Example of a series circuit with two consumers. Every device in the circuit reduces the voltage that is available for the remaining ones. This voltage can be calculated with the help of Ohm’s Law, the so-called “voltage drop” of each device. The sum of all voltage drops in a series circuit has to be equal to the applied potential difference. With the help of a series circuit it is possible to power several devices which need the same amount of current, whereas the voltage may vary. A.7.2. Parallel Circuit Georg Gerstweiler An electronic circuit is called parallel circuit, if each component has a separate connection to the voltage source. That means the current for every device flows through its own path without passing another device from a parallel path (Figure 70). Figure 70: Parallel Circuit Appendix Gerstweiler, Vonach, 2011 123 In contrast to serial circuits not the current stays the same over the whole circuit, but the voltage does. This makes it possible to use several devices which need the same potential difference at once. This is done in every household over the world, where every lamp operates at 120 V respectively 230 V in parallel. Because of the constant voltage there are parts of the circuit, like in Figure 70 between the points A and B respectively C and D, which have to transport the current of the whole circuit. The calculation of the power dissipation in a parallel circuit equals the sum of the power used in every branch. With the help of the formula in appendix A.4 the power dissipation of each part of the circuit in Figure 71 can be calculated and summed up for the total dissipation of 30 W. Figure 71: Power Dissipation in Parallel Circuits A.7.3. Series-Parallel Circuit Georg Gerstweiler There are situations in circuits where some devices need the same voltage but others need the same current. This can be solved by a “series-parallel circuit”, which is a combination of both, like it is shown in the example of Figure 72. This circuit has a series branch of two resistors and a parallel branch with just one resistor. While the current at R1 and R2 is 1 A at different voltages, R3 has 2 A. Figure 72: Series-Parallel Circuit Appendix 124 Gerstweiler, Vonach, 2011 B. Motion Suit Development Software Georg Gerstweiler In order to implement the architecture described in chapter 6.3 and to design the necessary hardware, different software packages are available. In the following a brief introduction of the programming environments that were used to implement the user interface software and the firmware that is intended to run on the ATmega168. Furthermore the third chapter presents the application for designing the basic breakout board, which is routing the paths in the control box. B.1. Processing IDE Georg Gerstweiler In order to set individual frames, which represent the actually available marker patterns, the authors implemented an application with a GUI to handle the communication between a workstation and the motion suit. It allows the user to comfortably choose the desired markers to be turned on or off. In addition it has to be capable of transmitting commands to the motion suit control box via a serial communication channel. In this project “Processing” [Processing '11] in the version 1.0.6 is used to implement this feature. It is an open source programming language inspired by Java and supports the developer especially in fast prototyping, but also allows the implementation of complete applications. An environment to run and compile the source code also known under the same name is provided over the webpage as well. The main idea for using this language was to allow the creation of images, animations and user interactions in an uncomplicated way, which makes it perfect for the implementation of the required user interface software. Appendix Gerstweiler, Vonach, 2011 125 B.2. Arduino IDE Emanuel Vonach To write software for the Arduino board presented in chapter 6.2.2, an open-source development environment also called “Arduino” [Arduino '11] (version 0017) is provided by the associated community over their webpage for free and for all common operating systems. It is intended to be used not only by professionals but also by artists and hobbyists. Therefore it is very easy to write code for the on-board chip. Because of the infrastructure already provided by the Arduino board the implemented software can be instantaneously deployed without the need for additional hardware. The Arduino programming language is deviated from “Wiring”, which is a comparable electronics prototyping platform based on the Processing environment. For that reason the development environment also looks very similar. Although the Arduino and Processing languages are related, there are a number of differences, since Arduino is implemented in C/C++ while Processing is based on Java. One of the major advantages of the Arduino environment over other microcontroller platforms for physical computing is the successful balance between a simple and comprehensible environment for amateurs and high flexibility for advanced users. Due to the fact that it is open source, it can easily be extended with customary C/C++ libraries, and since the Arduino platform is based on an ATMEL chip, AVR C code (a set of instructions especially for those ICs) is supported as well. Another big advantage is the large community around the Arduino platform, because of which a lot of helpful information and support is available on the internet. Furthermore since the Arduino boards are employed to interface with a diversity of devices, plenty of complete libraries or code examples can be found for that purpose. B.3. EAGLE Light Edition Georg Gerstweiler To keep the control box for the motion suite compact, it is beneficial to mount all the devices described in chapter 6.2 on a “printed circuit board” (PCB). In order Appendix 126 Gerstweiler, Vonach, 2011 to create the corresponding circuit diagram the software package “EAGLE” (Easily Applicable Graphical Layout Editor) by the company CadSoft Inc [CadSoft '11] can be used. The software is compatible with Windows, Linux and Mac OS and is also available as freeware version, the “Light Edition”. Because the PCB for the control box should be as small as possible, all components must be carefully arranged. In order to achieve this, the freeware version of EAGLE (version 5.6.0) is used, which supports the designing of boards with a maximum dimension of 100 mm by 80 mm. This size equates to a half “Eurocard”, which is a standardized format for PCBs. The freeware version is not only limited in the maximum size of the board, but also does not allow more than two layers, namely bottom and top. Furthermore the user can only create one design in each project. Beside these limitations the EAGLE Light Edition is fully functional in comparison to the full version and this makes it perfect to use in the context of this work. Figure 73: EAGLE Schematic Editor Figure 74: EAGLE Layout Editor EAGLE is a package that combines three main modules in one application for designing a PCB layout. These are the “Schematic Editor”, the “Layout Editor” and the “Autorouter”. The Schematic Editor helps to plan the circuit diagram by allowing the user to place the schematic representations of the devices per drag and drop in well-arranged positions. Although the library of EAGLE contains plenty of different predesigned electric components, it may sometimes be required to create a new schematic and board layout of a used piece of hardware. After arranging the components on the schematic, the required interconnections have to be defined either directly with a line tool or just by naming the corresponding pins equally. In this editor it is not necessary to pay Appendix Gerstweiler, Vonach, 2011 127 any attention to compactness, because the focus lies on creating a well understandable overview (Figure 73). As a next step it is required to put all the devices on a virtual circuit board with the help of the layout editor (Figure 74). In this editor the different components are visualized with their boundaries including the required solder pads, which are necessary to establish a connection with the board. It is not necessary to consider all the complex connections between the individual contacts, because the already mentioned Autorouter takes care of routing the conductive paths in between. This algorithm also optimizes the wiring according to distance and the number of crossings. Different colors are signalizing in which layer each path or soldering pad is located. Another feature of the application is a special test that can be performed called “Electrical Rule Check” (ERC), which helps to pinpoint possible errors in the designed circuitry. Appendix 128 Gerstweiler, Vonach, 2011 C. Chapter - Author Correlation 1. Introduction  Emanuel Vonach  1.1. Introduction to Virtual Reality  Georg Gerstweiler  2. Main Objectives  Emanuel Vonach  2.1. Central Hypothesis  Emanuel Vonach  2.2. Research Objectives  Emanuel Vonach  2.3. Limitations  Emanuel Vonach  3. Related Work  Emanuel Vonach  3.1. Motion Tracking Technologies  Emanuel Vonach  3.1.1. Mechanical Tracking Emanuel Vonach  3.1.2. Magnetic Tracking Emanuel Vonach  3.1.3. Acoustic Tracking Emanuel Vonach  3.1.4. Inertial Tracking Emanuel Vonach  3.1.5. Optical Tracking Emanuel Vonach  3.1.6. Hybrid Tracking Emanuel Vonach  3.2. Concepts Related to Education and Rehabilitation Applications  Georg Gerstweiler  3.3. Analysis and Motivation  Georg Gerstweiler  4. Discussion of Requirements and Technologies  Georg Gerstweiler  4.1. Requirements of a tracking system for teaching motion skills  Georg Gerstweiler Appendix Gerstweiler, Vonach, 2011 129 4.2. Tracking Technology Decision  Georg Gerstweiler  4.3. VR-Setup  Georg Gerstweiler  4.4. Skeleton Estimation Algorithm  Emanuel Vonach  4.5. Marker Layout  Emanuel Vonach  5. Electronic Fundamentals and Communication Protocols  Georg Gerstweiler  5.1. Light Emitting Diode  Emanuel Vonach  5.2. LED Matrix Circuit  Emanuel Vonach  5.3. Galvanic Coupling  Emanuel Vonach  5.4. Serial Peripheral Interface – SPI  Georg Gerstweiler  5.5. Inter-Integrated Circuit – I²C  Emanuel Vonach  6. Motion Suit - Conceptual Design  Georg Gerstweiler  6.1. Material  Georg Gerstweiler  6.1.1. Base Material Emanuel Vonach  6.1.2. Conductive Thread Georg Gerstweiler  6.1.3. Insulation Emanuel Vonach  6.1.4. Connectors Emanuel Vonach  6.1.5. IR-Emitter Georg Gerstweiler  6.1.6. Emission Regulation Emanuel Vonach  6.2. Control Box Components  Georg Gerstweiler  6.2.1. LED Matrix Driver Emanuel Vonach  Appendix 130 Gerstweiler, Vonach, 2011 6.2.2. Controlling Unit Georg Gerstweiler  6.2.3. Power Source and Consumption Georg Gerstweiler  6.2.4. Additional Components Emanuel Vonach  6.3. Software Architecture for Operation and Management  Georg Gerstweiler  7. Motion Suit – Construction  Emanuel Vonach  7.1. Feasibility Tests  Georg Gerstweiler  7.2. Suit Adaptations  Emanuel Vonach  7.3. Wiring  Emanuel Vonach  7.4. Connectors and Connections  Emanuel Vonach  7.5. Active Markers  Georg Gerstweiler  7.6. Control Box  Georg Gerstweiler  7.7. Software for Operation and Management  Emanuel Vonach  7.7.1. Firmware Emanuel Vonach  7.7.2. Motion Suit Manager Georg Gerstweiler  7.8. Costs  Georg Gerstweiler  10. Evaluation of the Motion Suit  Emanuel Vonach  10.1. Setup of User Tests  Georg Gerstweiler  10.2. User Test Observations  Emanuel Vonach  10.3. Results  Emanuel Vonach  11. Conclusion and Future Work  Georg Gerstweiler  Appendix Gerstweiler, Vonach, 2011 131 Appendix  A. Electrical Fundamentals  Emanuel Vonach  A.1. Voltage (V)  Georg Gerstweiler  A.2. Current (A)  Georg Gerstweiler  A.3. Resistance (R)  Georg Gerstweiler  A.4. Electric Power (P)  Georg Gerstweiler  A.5. Ohm’s Law  Georg Gerstweiler  A.6. Capacitor  Georg Gerstweiler  A.7. Series and Parallel Circuits  Georg Gerstweiler  A.7.1. Series Circuit Georg Gerstweiler  A.7.2. Parallel Circuit Georg Gerstweiler  A.7.3. Series-Parallel Circuit Georg Gerstweiler  B. Motion Suit Development Software  Georg Gerstweiler  B.1. Processing IDE  Georg Gerstweiler  B.2. Arduino IDE  Emanuel Vonach  B.3. EAGLE Light Edition  Georg Gerstweiler  C. Chapter - Author Correlation  Bibliography  Appendix 132 Gerstweiler, Vonach, 2011 Table of Figures Figure 1: VR System Components  Figure 2: Animazoo Gypsy 7, a mainly mechanical motion capture system (courtesy of Animazoo).  Figure 3: “MotionStar” by Ascension Technology Corporation (courtesy of Ascension).  Figure 4: a. Logitech “Head Tracker”, b. Tracking Volume (both [Logitech '92])  Figure 5: Xsens “MVN” is an inertial motion capture solution (courtesy of Xsens).  Figure 6: Schematic of the Iotracker System  Figure 7: A hybrid suit used for the television series “Sid the Science Kid” (courtesy of The Jim Henson Company).  Figure 8: A VR application applied in sports using passive markers [Bideau '04].  Figure 9: MoCap Suit with Inertial Sensors [Fitzgerald '07].  Figure 10: MoCap Shirt LED Markers [Staples '06]  Figure 11: Active Markers for Kinematic Measurements [Kunju '09]  Figure 12: Complete Active MoCap Suit [Kirk '05]  Figure 13: The estimation of a skeleton: a. one of the authors moving all his joints, b. the marker positions provided by the tracking system, c. the grouped markers, d. the fitting skeleton  Figure 14: The marker layout for the motion suit.  Figure 15: Schematic of a diode: without power supply a. electrons fill the holes and b. create a depletion zone, c. if a power supply is connected this zone dissipates.  Figure 16: Only the LEDs of a matrix circuit that are connected to positive potential and ground in the same time slot are active.  Figure 17: Due to the galvanic coupling between circuits I and II, the voltage uC drops at the common impedance ZC and a signal U2 may be disturbed.  Figure 18: SPI Functional Principle  Figure 19: SPI Communication Sample [Maxim '09]  Figure 20: I²C Communication Procedure  Figure 21: Motion Suit Material Overview  Figure 22: This track suit and headband are a suitable base material for the motion suit.  Figure 23: Spool of Conductive Thread  Figure 24: This graph shows the conductivity of the thread measured with a digital circuit analyzer, when using multiple threads for the same connection over a distance of 10 cm.  Figure 25: a. Test Setup for Insulation Material, b. Glue for Napkin Decoupage  Figure 26: a. “Micro MATE-N-LOK”, b. “Wiremount Socket” and c. IDC Header with Ribbon Cable  Figure 27: SMD Version of the IR-LED HSDL-4400  Figure 28: Relative Radiant Intensity vs. Angular Displacement [HewlettPackard]  Appendix Gerstweiler, Vonach, 2011 133 Figure 29: Peak Forward Current vs. Forward Voltage [HewlettPackard]  Figure 30: Comparison between a. passive reflective and b.-i. active IR-LED markers at a distance of 5 m.  Figure 31: a. Test Setup in a Furniture Store, b. Possible Emission Regulation Pad  Figure 32: a. Photo and b. Schematic of the MAX6954 LED Matrix Driver by Maxim [Maxim '09]  Figure 33: Arduino Nano  Figure 34: a. “LiPo Saver” by Modelcraft, b. Voltage Regulator, c. Toggle Switch  Figure 35: Motion Suit Control Architecture  Figure 36: First test setup with Arduino NG, MAX7219 and standard 5 mm LEDs.  Figure 37: Second test setup with Arduino Nano, MAX6954, HSDL-4400 and a resistance between 20 Ω and 150 Ω in series with each LED.  Figure 38: Iotracker camera image of second test setup.  Figure 39: First prototype on clothing with connector and HSDL-4400.  Figure 40: Adaptation for the Hand  Figure 41: a. The heel piece, b. the instep piece and c. the strap to fixate them on the foot.  Figure 42: Strap Construction for the Feet  Figure 43: Four of these loops are mounted on the trousers and can be pulled through slots in the shirt.  Figure 44: The cord can be knotted over the belly for additional solidity.  Figure 45: a. Bridgeovers with Parallel Seams, b. Applied Insulation, c. Contacts Pulled to the Outside  Figure 46: Bridgeovers for the Upper Body  Figure 47: Patch to Reinforce Regions with Multiple Crossings  Figure 48: Bridgeover for Joint LEDs a. Before and b. After Mounting  Figure 49: Cables of Conductive Thread with Heat Shrinkable Tubing as Insulation  Figure 50: The Connection to the Ribbon Cables  Figure 51: First LED Assembly  Figure 52: Components for Marker Assembly  Figure 53: Final Marker Assembly  Figure 54: Marker Assembly including Foam Craft Patch  Figure 55: Individual Components of the Control Box  Figure 56: PCB Layout  Figure 57: Finally Assembled Control Box  Figure 58: Control Box Schematic  Figure 59: Motion Suit Manager – User Interface for Marker Layouts  Figure 60: Screenshot of MoCapGym during a training session in comparison mode.  Figure 61: Participant During Training Session  Figure 62: The developed active motion suit in combination with a passively tracked head mounted display.  Figure 63: Potential Difference  Figure 64: Electric Conductor  Appendix 134 Gerstweiler, Vonach, 2011 Figure 65: Resistor Symbols: a. DIN, b. ANSI, c. Resistor Color Coding  Figure 66: Types of Resistors: a. Metal Film Resistor, b. Surface Mount, c. Variable Resistor – Potentiometer  Figure 67: Simple Electronic Circuit  Figure 68: Structure of Capacitors  Figure 69: Example of a series circuit with two consumers.  Figure 70: Parallel Circuit  Figure 71: Power Dissipation in Parallel Circuits  Figure 72: Series-Parallel Circuit  Figure 73: EAGLE Schematic Editor  Figure 74: EAGLE Layout Editor  Bibliography Gerstweiler, Vonach, 2011 135 Bibliography [3M '11] 3M Electronics Solutions Division. (2011, accessed: 12. September 2011). "3M Wiremount Socket". Available: http://multimedia.3m.com/mws/mediawebserver?mwsId=SSSSSu7zK 1fslxtU4x2vNx_xev7qe17zHvTSevTSeSSSSSS--&fn=ts0718.pdf [3x3 '11] 3x3 Designs. (accessed: 12. September 2011). "3x3.ca". Available: http://www.3x3.ca/ [Akenine-Möller '02] T. Akenine-Möller and E. Haines, "Real-time Rendering", 2 ed. Wellesley, Mass.: A K Peters, 2002. [Animazoo '11] Animazoo Uk Ltd. (accessed: 12 September). "Animazoo Motion Capture Systems and Technology; Gypsy-7". Available: http://www.animazoo.com/motion-capture-systems/gypsy-7-motion- capture-system/ [Arduino '11] Arduino. (accessed: 12. September 2011). "Arduino - HomePage". Available: http://arduino.cc/ [Ascension '04] Ascension Technology Corp. (2004, accessed: 12. September 2011). "MotionStar Real-Time Motion Capture". Available: http://www.ascension-tech.com/docs/products/motionstar_10_04.pdf; http://www.ascension-tech.com [Bideau '04] B. Bideau, F. Multon, R. Kulpa, L. Fradet, and B. Arnaldi, "Virtual Reality Applied to Sports: Do Handball Goalkeepers React Realistically to Simulated Synthetic Opponents?", in Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry, Singapore, pp. 210-216, 2004. [Bowman '05] D. A. Bowman, E. Kruijff, J. Joseph J. LaViola, and I. Poupyrev, "3D User Interfaces: Theory and Practice" vol. 1. Boston: Addison-Wesley, 2005. Bibliography 136 Gerstweiler, Vonach, 2011 [Burdea '03] G. Burdea and P. Coiffet, "Virtual Reality Technology", 2 ed. Hoboken, N.J.: Wiley-Interscience, 2003. [CadSoft '11] CadSoft Inc. (accessed: 12. September 2011). "CadSoft EAGLE PCB Design Software". Available: http://www.cadsoftusa.com/eagle- pcb-design-software/ [CMU '07] Carnegie Mellon University Graphics Lab. (2007, accessed: 12. September 2011). "Marker Placement Guide". Available: http://mocap.cs.cmu.edu/markerPlacementGuide.pdf [CreaDirect '11] CreaDirect GmbH. (accessed: 12. September 2011). "Creadirect - Produkte; Ideen - Relief-Facetten-Lack, Textiltransfer- Gel ...". Available: http://www.creadirect.eu/produkte- ideen/seite12.html [Ellis '89] S. Ellis and A. Grunwald, "Visions of Visualization Aids: Design Philosophy and Observations", in SPIE OE/LASE Symposium on three-dimensional visualization of scientific data, Bellingham, Washington, pp. 220-227, 1989. [Fitzgerald '07] D. Fitzgerald, J. Foody, D. Kelly, T. Ward, C. Markham, J. McDonald, and B. Caulfield, "Development of a Wearable Motion Capture Suit and Virtual Reality Biofeedback System for the Instruction and Analysis of Sports Rehabilitation Exercises", presented at the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS 2007), 2007. [Freescale '10] Freescale Semiconductor. (2010, accessed: 12. September 2011). "MC9S12KG128 Data Sheet" (1.16 ed.). Available: http://www.freescale.com/files/microcontrollers/doc/data_sheet/MC9S 12KG128_V1.pdf [Gordon '09] D. Gordon and E. Robertson. (2009, accessed: 12. September 2011). "Vicon Workstation Quick Reference Guide". Available: http://www.health.uottawa.ca/biomech/courses/apa4311/Vicon%20W orkstation%20Quick%20Reference%20Guide.pdf [Goudas '95] M. Goudas, S. Biddle, K. Fox, and M. Underwood, "It Ain't What You Do, It's the Way That You Do It! Teaching Style Affects Bibliography Gerstweiler, Vonach, 2011 137 Children's Motivation in Track and Field Lessons," The Sport Psychologist, vol. 9, pp. 254–264, 1995. [Grob '99] B. Grob, "Basic Electronics", 8 ed. New York, NY: Glencoe McGraw- Hill, 1999. [Harris '02] T. Harris and W. Fenlon. (2002, accessed: 12. September 2011). "How Light Emitting Diodes Work". Available: http://www.howstuffworks.com/led.htm [Harting] Harting. (accessed: 12. September 2011). "08. SEK IDC Connectors". Available: http://www.harting.com/imperia/md/content/lg/hartingusa/service/cata logdownload/devicecon/devicecon_gb_chapt_08_g.pdf [Henson '11] The Jim Henson Company. (accessed: 12. September 2011). "The Jim Henson Company's Creature Shop - Jim Henson's Sid the Science Kid". Available: http://www.creatureshop.com/sid.php [HewlettPackard] (accessed: 12. September 2011). "High-Performance IR Emitter and IR PIN Photodiode in Subminiature SMT Package - HSDL-4400". Available: http://www.datasheetcatalog.org/datasheet/hp/HSDL-5420.pdf [Irazabal '03] J.-M. Irazabal and S. Blozis, "I2C Bus Overview", in DesignCon, San Jose, CA, 2003. [ITW] ITW Pancon. (accessed: 12. September 2011). "LAT-CON Series 057 Low Profile IDC Headers". Available: http://www.itwpancon.com/LATCON%20pages/LATCON_series057lo w.pdf [Jeff '09] A. Jeff. (2009, accessed: 12. September 2011). "NBA LIVE Motion Capture". EA Games: NBA LIVE Motion Capture. Available: http://www.easports.com/news/item/file/NBA-LIVE-Motion-Capture [Jordan '02] J. Jordan, "LED Matrix Display Driver," Banks of the Boneyard, The Journal for the Association for Computing Machinery at the University of Illinois in Urbana-Champaign, vol. 20, 31. January 2002. Bibliography 138 Gerstweiler, Vonach, 2011 [Kirk '05] A. G. Kirk, J. F. O'Brien, and D. A. Forsyth, "Skeletal Parameter Estimation from Optical Motion Capture Data", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2005), San Diego, CA, pp. 782-788, 2005. [Kunju '09] N. Kunju, N. Kumar, D. Pankaj, D. P. Singh, and A. Kumar, "Algorithm for Kinematic Measurements using Active Markers", presented at the IEEE International Advance Computing Conference (IACC 2009) Patiala, India, 2009. [Langguth '06] European Copper Institute, Hochschule für Technik und Wirtschaft and Copper Development Association. (2006, accessed: 12. September 2011). "Power Quality Application Guide". 6.1.2. Available: http://www.copperinfo.co.uk/power- quality/downloads/pqug/612-fundamentals-of-electromagnetic- compatibility.pdf [Leens '09] F. Leens, "An Introduction to I²C and SPI Protocol," Instrumentation & Measurement Magazine, IEEE, vol. 12, February 2009. [Logitech '92] Logitech, Inc. (1992, accessed: 12 September 2011). "3D Mouse & Head Tracker Technical Reference Manual". Available: http://www.vrdepot.com/manual-tracker.pdf [Maxim '03] Maxim Integrated Products, Inc. (2003, accessed: 12. September 2011). "Maxim MAX7219/MAX7221" (4 ed.). Available: http://datasheets.maxim-ic.com/en/ds/MAX7219-MAX7221.pdf [Maxim '09] Maxim Integrated Products, Inc. (2009, accessed: 12. September 2011). "Maxim MAX6954" (4 ed.). Available: http://datasheets.maxim- ic.com/en/ds/MAX6954.pdf [Miaw '10] D. Miaw, "Second Skin: Motion Capture with Actuated Feedback for Motor Learning", Master of Engineering in Electrical Engineering and Computer Science, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Massachusetts, 2010. Bibliography Gerstweiler, Vonach, 2011 139 [Microsoft '11] Microsoft Corporation, Harmonix Music Systems, Inc. (accessed: 12. September 2011). "Dance Central". Available: http://www.dancecentral.com [Mirabella '11] O. Mirabella, A. Raucea, F. Fisichella, and L. Gentile, "A Motion Capture System for Sport Training and Rehabilitation", presented at the 4th International Conference on Human System Interactions (HSI), 2011. [Pintaric '07] T. Pintaric and H. Kaufmann, "Affordable Infrared-Optical Pose- Tracking for Virtual and Augmented Reality", presented at the Trends and Issues in Tracking for Virtual Environments, Charlotte, NC, USA, 2007. [PMDTechnologies '11] PMDTechnologies GmbH. (accessed: 12. September 2011). "PMDTec: PMD[vision]® CamCube 3.0". Available: http://www.pmdtec.com/products-services/pmdvisionr- cameras/pmdvisionr-camcube-30/ [PrimeSense '11] PrimeSense, Ltd. (accessed: 12. September 2011). "PrimeSense Natural Interaction". Available: http://www.primesense.com/en/technology [Processing '11] Processing. (accessed: 12. September 2011). "Processing.org". Available: http://processing.org/ [Schönauer '07] C. Schönauer, "Skeletal Structure Generation for Optical Motion Capture", Dipl.Ing., Institute of Software Technology and Interactive Systems, Vienna University of Technology, Vienna, 2007. [Schönauer '11] C. Schönauer, T. Pintaric, and H. Kaufmann, "Full Body Interaction for Serious Games in Motor Rehabilitation", in Proceedings of the 2nd Augmented Human International Conference, New York, NY, USA, pp. 1-8, 2011. [SGS '93] STS-Thomson Microelectronics Group of Companies. (1993, accessed: 12. September 2011). "L78S00 Series 2A Positive Voltage Regulators". Available: http://www.datasheetcatalog.org/datasheet/SGSThomsonMicroelectr onics/mXrvsqv.pdf Bibliography 140 Gerstweiler, Vonach, 2011 [Smith '11] (accessed: 12. September 2011). "Lamé Lifesaver - Conductive Thread for Everything". Available: http://members.shaw.ca/ubik/thread/thread.html [Staples '06] J. A. Staples and T. A. Davis, "Motion Capture for the Masses," INFOCOMP Journal of Computer Science, vol. 5, pp. 18-24, 2006. [TE '07] Tyco Electronics Corporation. (2007, accessed: 12. September 2011). "AMP Soft Shell Pin and Socket Connectors" (4 ed.). Available: http://www.tycoelectronics.com/commerce/DocumentDelivery/DDEC ontroller?Action=showdoc&DocId=Catalog+Section%7F82181_SOF TSHELL_HIGH_DENSITY%7F0508%7Fpdf%7FEnglish%7FENG_C S_82181_SOFTSHELL_HIGH_DENSITY_0508.pdf%7F1-1445022-0 [Xsens '10] Xsens Technologies B.V. (2010, accessed: 12. September 2011). "MVN Inertial Motion Capture". Available: http://www.xsens.com/images/stories/products/PDF_Brochures/mvn %20leaflet.pdf