Federated Reinforcement Learning for Efficient Mobile Crowdsensing Under Incomplete Information

Abstract

Mobile crowdsensing (MCS) is a distributed sensing architecture that utilizes existing sensors on mobile units (MUs) to perform sensing tasks. An MCS platform (MCSP) publishes the sensing tasks and the MUs decide if they want to participate in their execution in exchange for money. The MCS system is characterized by its dynamic nature in which the task requirements, the MUs’ availability, and their available resources change over time. The MUs aim to find an efficient task participation strategy to maximize their income, while the MCSP focuses on maximizing the number of completed tasks. As optimal task participation strategies require perfect noncausal information about the MCS system, which is unavailable in realistic scenarios, the main challenge in MCS is to find an efficient task participation strategy for the MUs under incomplete information. To this aim, a novel fully decentralized federated deep reinforcement learning algorithm, termed FDRL-PPO, is proposed. FDRL-PPO enables every MU to learn its own task participation strategy based on its experiences, available resources, and pReferences, without relying on perfect noncausal information about the MCS system. To replenish their batteries, the MUs rely on energy harvesting. As a result, their available energy varies over time, leading to varying availability and fragmented learning experiences. To mitigate these challenges, the proposed approach leverages federated learning, enabling MUs to collaboratively improve their models without having to share private raw data like their own experiences. By exchanging only learned models, MUs collectively compensate for individual limitations and find more scalable, robust, and efficient task participation strategies. Comprehensive evaluations on both synthetic and real-world datasets show that FDRL-PPO consistently outperforms benchmark algorithms in terms of task completion ratio, fairness in task completion, energy consumption, and number of conflicting proposals.