CRiSE 1-3-7 - compilations of real-world inspired robotic task simulation environments for meta-reinforcement learning

Abstract

Learning new robotic tasks from scratch is a time-consuming and sophisticated process.Meta-Reinforcement Learning (MRL) is a method to extract and preserve generalised information about learning how to reinforcement learn a set of robotic training tasks.Based on this information, previously unseen tasks are successfully learned in just a few trials. Although this approach is promising in terms of making robots learn new tasks more efficiently and adapt their task-fulfilling strategy faster, simulation environments that foster MRL by providing sufficiently broad robotic task sets are still a rare commodity. This thesis introduces three compilations of real-world inspired robotic task simulation environments, abbreviated CRiSE 1-3-7, that serve as simulation platforms to execute and test MRL algorithms. CRiSE 1-3-7 is designed to mimic the setting in the Automation and Control Institutes Robotic Lab at TU Wien. Here, the goal is to lower the barriers of getting meta-reinforcement learned policies from simulation to real world.For better usability and future modularity, CRiSE 1-3-7 is integrated into the experiment structure of an established reinforcement learning toolkit. Experiments conducted with two state-of-the-art MRL algorithms prove that CRiSE 1 and 3 are at least partially learnable with the two baseline algorithms. The trained policies demonstrate a lack of generalisation during meta-training and fail few-shot adaptation on CRiSE 7. Overall,CRiSE 1-3-7 constitutes promising robotic task compilations for future MRL algorithm development and benchmarking. CRiSE 7 is especially a challenging benchmark for futureMRL algorithms because of its complex structure.All experiment launchers and the code base of CRiSE 1-3-7 is open-source and available at the author’s GitHub webpage (https://github.com/jivancsics/robosuite_CRiSE137).