Resource Allocation for UAV Swarm-Assisted Green ISAC Networks via Multi-Agent RL

Integrated sensing and communication (ISAC) technology based on unmanned aerial vehicles (UAVs) has recently been recognized as an indispensable functionality for the upcoming Sixth Generation (6G) green wireless networks. However, in harsh environments such as rural isolated areas, the resource allocation of sensing-communication service for UAVs, still seriously affects the ISAC network performance due to its inevitable defect of limited power energy. Different from previous studies, we propose a sensing-communication resource allocation analytical framework, which innovatively focuses on utilizing reinforcement learning (RL) to provide sensing prior information for UAVs, thereby obtaining satisfactory sensing and communication services for ground terminals (GTs). We first analyze the effects of resource allocation on communication and sensing respectively. On this basis, we form a sensing-communication optimization problem from the perspective of RL, specifically maximizing the total energy efficiency of the UAVs while providing sensing prior information. To solve this composite optimization problem, we first propose an improved FBSS algorithm to initialize the Q-table for UAVs for sensing purpose, and further develop a distributed Q-learning based scheme that enables each UAV to discover the optimal strategy for maximizing its expected reward. Compared with the state-of-the-art benchmarks, the numerical results show that the communication performance of the proposed method is improved by more than 40% on average. In addition, this scheme can enhance the sensing accuracy of the studied network by more than 20% without sacrificing the communication capability.