iChores: intuitive collaboration with household robots in everyday settings

Abstract

In this article, we present iChores, an open-source, robot-agnostic interface designed to facilitate intuitive human interaction for household tasks, enabling users to direct robots through natural language and deictic gestures (pointing) to per- form complex chores, such as tidying a room. The system utilizes a modular microservice architecture that integrates state-of-the-art vision, natural language understanding, and reasoning modules. We detail the implementation of the framework, provide a public repository for reproducibility, and present an evaluation of 30 sce- narios of increasing complexity. The general architecture of iChores and its various modules are briefly described. The implementation is publicly available through our GitHub project page, where all modules are provided as their own repositories.