Enhancing the Supervision of Out-of-View Robots: A Study on Multimodal Feedback and Monitoring Screens

Abstract

Objective: investigating the effect of two support methods (multimodal feedback, monitoring screens, and a combination of both) on human dual-task performance, cognitive workload, and user experience when supervising an out-of-sight autonomous robot. Method: A 2x2 within-group user study was conducted in VR with 26 participants involving a cognitive-cognitive dual-task setting. Participants had to simultaneously solve math problems and supervise the robot. Different support methods were provided: multimodal feedback, a screen showing real-time robot activity, and a combination of both. Objective performance metrics and subjective feedback on cognitive load and user experience were collected using standard questionnaires. Data were statistically analyzed, and thematic analysis was performed on post-study debriefing interviews. Results: The support methods improved overall user experience and positively impacted robot collaboration performance while decreasing math task performance. Cognitive load was unaffected. Multimodal feedback with a monitoring screen was perceived as the most helpful. Conclusion: The results suggest that multimodal feedback can improve user experience and improve supervision, but may partially decrease primary task performance. The findings highlight the importance of examining the effect of support methods in specific situations, depending on task priority.