Sim-to-Real: Tiny Deep Learning Agents on Resource-Constrained Embedded Microcontrollers

Abstract

Deep Reinforcement Learning offers a powerful approach for developing advanced control policies based solely on plant model simulations. However, deploying these policies on industrial-scale embedded microcontroller systems presents significant challenges. Imperfect plant models, parameter uncertainty, and modeling errors can compromise robust control operation, while the limited computational power of realtime microcontrollers necessitate adaptations to ensure efficient execution of learned policies. In this work, we present a reinforcement learning-based motor control concept and investigate the impact of compute-efficient deployment techniques. We evaluate the effects of quantization on the control policy, providing key insights into how it influences long short-term memory (LSTM) cell behavior in control problem settings, and further explore the associated deployment challenges through experiments on a real-world motor control application.