Deep Reinforcement Learning for Cell-Free Massive MIMO Network Optimization

Abstract

Despite the significant advancements in wireless communication technologies, intercell interference remains a limiting factor due to the cell-centric design of traditional mobile networks. Cell-free massive multiple-input multiple-output (MIMO) is a paradigm shift in network architecture, where we replace the fixed cell boundaries with a seamless network of cooperating access points (APs) to achieve a uniformly good performance throughout the coverage area. To harness its full potential, it is necessary to address its scalability issue and the need for dynamic optimization based on the current state of the wireless environment. Compared to conventional optimization techniques and (un-)supervised machine learning, deep reinforcement learning (DRL) is capable of operating model-free, without requiring any prior knowledge, including training datasets, and in an online manner, making it an effective tool for real-time network adaptation. Motivated by these advantages, this dissertation leverages DRL for the realization of scalable, self-adapting cell-free massive MIMO. The dissertation consists of three main parts. The first part focuses on user-centric clustering, where each user equipment (UE) is served by only a subset of APs. The second part of the dissertation capitalizes on single-agent RL (SARL) or cell-free network optimization. The last part of the dissertation promotes distributed learning architectures by employing multi-agent RL (MARL), in addition to SARL.