Lenssembly: Authoring Assembly Instructions in Augmented Reality Using Programming-by-Demonstration

Abstract

Managing the knowledge of assembly workers is crucial due to the valuable personal expertise of collected information over time that is hard to articulate. Unfortunately, the accumulated knowledge disappears when workers leave the company. Methods to record and transfer assembly knowledge between workers rarely exist due to the timeconsuming documentation of assembly steps. This paper presents Lenssembly, a mobile augmented reality system utilizing programming-by-demonstration to record, detect, and generate assembly instruction sequences using a head-mounted display. The assembly instructions are automatically detected using a neural network, preventing the need for manual documentation and time-intensive content creation for each assembly step. In a user study (N = 12) with two different assembly tasks, participants favored the recording functionality of Lenssembly while conducting fewer errors and perceiving less task load than traditional paper instructions. We discuss the implications of our results and conclude how technologies create repositories for storing and transferring expert worker knowledge.