Resource-Constrained Deep Post-Decision State Learning for Wireless Sensing System Scheduling

We explore uplink time scheduling for wireless multi-user energy-harvesting systems under a transmission time allocation budget constraint. We formulate the problem as a constrained Markov decision process (CMDP) and propose a novel post-decision state (PDS) based learning algorithm that decomposes the value function into analytically computable known costs and learned unknown costs. This decomposition exploits problem structure to accelerate convergence compared to standard reinforcement learning methods. An adaptive dual variable mechanism is integrated within the soft actor-critic framework to enforce time budget constraints dynamically. Simulations demonstrate rapid convergence and 12.5% performance improvement over standard actor-critic methods, with over 27.5% improvement relative to heuristic baselines and tight constraint satisfaction.