Priority belief propagation-based inpainting prediction with tensor voting projected structure in video compression

This paper presents a new video compression framework over H.264/AVC scheme that integrates our proposed structured priority belief propagation (BP)-based inpainting prediction (IP) to exploit the intrinsic nonlocal and geometric regularity in video samples. Unlike the existing edge-based inpainting adopted in lossy image coding, the optimal predictor could maintain the pixel-wise fidelity and the robust error resilience without any assistant information. Beyond the local prediction limitation of traditional intra and inter-modes, the priority BP with regularized structure priors of a spatio-temporal Markov random field is imposed on the predictor in an adaptive and more convergent sense. Specifically, the structured sparsity of the predicted macroblock region is inferred by tensor voting projected from the co-located decoded regions. In turn, the priority and visiting order of nodes are assigned according to the sets of updated beliefs as the propagation of messages. Through relatively few iterations of forward and backward process, the sparse inference of priority BP would ensure a stable marginal belief distribution on the structure and texture through updating local messages and beliefs. Within the optimal mode selection on rate-distortion optimization (RDO), the IP-mode with structured priority BP outperforms the existing vision-based approaches, and specially achieves a better objective rate-distortion performance besides visual quality. The IP-mode with structured priority BP can be applied to both I and P frames to generate low entropy residue, e.g., homogeneous visual patterns, and the computation complexity is also competitive with one iteration of sparse inference. Moreover, it behaves more resilient with an intrinsic probabilistic inference than the intra and inter-modes.