Deep Reinforcement Learning-Based Deferred Entanglement Path Selection in Quantum Networks

Conventional entanglement routing approaches decide the Entanglement Paths (EPs) to establish Entanglement Connections (ECs) before trying to create Entanglement Links (ELs). By doing so, very few EL failures will result in a low network throughput. In this paper, we study how to choose the EPs to establish ECs after knowing which ELs are successfully created. This is called the Deferred EP Selection (DEPS) problem. DEPS is a generalized integer multi-commodity flow problem and we cannot solve it quickly with conventional optimization methods. To address this issue, we propose a Deep Reinforcement Learning based EP Selection (DRLEPS) approach. The salient features of DRLEPS include (i) by controlling the number of candidate EPs, DRLEPS can achieve a trade-off between time complexity and the EC establishment rate; and (ii) using candidate EPs as input, DRLEPS is robust to request variation; and (iii) by training neural networks with different topologies, a model derived by DRLEPS can be applied to various networks (even with a different number of nodes) without fine-tune. Through extensive simulations, we show that even in a network with 200 nodes, DRLEPS can solve the DEPS problem in 0.39 seconds with a Nvidia GeForce 3090 GPU. It outperforms the approach always establishing ECs through the EP with the largest success probability by up to 23.4% in EC establishment rate. It also outperforms the Integer Linear Programming (ILP) based scheme, which can achieve the maximum EC establishment rate, by up to 184.2x in network throughput.