Elicitation and Evaluation of Mid-Air Hand Gestures for Global Earth Observation Data presented on Large Public Displays

Abstract

Global Earth observation data and therefrom derived information are essential for understanding the current state, natural processes and dynamics of the Earth system. However, the characteristics of these complex spatial data sets make accessibility to a lay audience challenging. Therefore, a prototype of a map viewer was developed as an interactive visualization tool to increase the approachability of these data sets. The prototype is designed to, on the one hand, display global satellite-based radar backscatter data and, on the other hand, be controllable by touchless mid-air hand gestures. The definition of an agreed upon set of touchless mid-air hand gestures is still an open question, especially in the context of interacting with complex data sets. To determine the base functions and features of such a novel system, a focus group discussion was conducted. In total, seven participants including the moderator discussed about existing online web maps and their advantages and disadvantages. Further major topics were how satellite-based radar backscatter data could be explained and presented to novices as well as how touchless hand gestures are currently perceived and could improve the interaction with such data. Based on the results of the focus group discussion, a gesture elicitation study with 30 participants was conducted. With individually held online meetings, test persons proposed gestures for twelve different tasks. In total, 462 gesture proposals as well as subjective ratings and qualitative statements of the mental creation process were gathered. From this elicitation touchless mid-air hand gestures and common narratives among the test persons were obtained. Quantitatively, for navigational map tasks which includes zooming and panning, the consensus could be clearly defined with the calculated agreement rates. For all other referents, the qualitative analysis which also took the second and even third proposals into account, gave suggestions for common narratives. Then, the consensus set of gestures and narratives as well as the desirable features obtained from the focus group discussion, were basis of the prototype implementation. The Leap Motion Controller (LMC) was used to determine the position and characteristics of the end-user’s hands within its field of view. The satellite-based radar backscatter data derived by Bauer-Marschallinger et al. [5] was used as the base map. The emerging system was planned to be located in a public space in the premises of the Department of Geodesy and Geoinformation at the TU Wien. Therefore, the usability and the user experience of the prototype was tested with ten participants. With an average score of 75.75% of the System Usability Scale (SUS), the prototype already reached a positive overall tendency of the usability. Furthermore, the User Experience Questionnaire (UEQ) also indicated a positive tendency especially for the scales Attractiveness, Stimulation and Novelty. Whereas, the lower values for Perspicuity, Efficiency and Dependability suggested that improvements of the prototype were needed. As a last step, the discovered issues were countered with solutions of the implementation, such as redesigning the tutorial or a rethought realization of the semantics of features.