Transformer DNNs for Versatile CSI-Based Wireless Communication and Localization

Abstract

Transformer deep neural networks (DNNs) form the backbone of today’s highly successful natural language processing systems. In so-called foundation models, transformers are employed to extract high-level features that enable various downstream tasks. Building on our previously developed wireless transformer (WiT) for channel state information (CSI)-based localization, we extend the same basic WiT architecture in this work to handle also other tasks in wireless systems using CSI. This suggests that WiT could be a suitable candidate architecture for a wireless foundation model. Specifically, we develop a generic WiT architecture and train it for multi-user beamforming and multicarrier power allocation.