3D Convolutional Generative Adversarial Networks for Detecting Temporal Irregularities in Videos

In this work, we introduce a novel method for video temporal irregularity detection using the discriminative framework of 3D convolutional generative adversarial networks (3D-GANs). Temporal irregularities indicate unusual video segments. Detecting such irregularities is essential to video analysis applications like video anomaly detection and video summarization. To detect temporal irregularities in videos we need to address two problems: 1) temporal irregularities are difficult to define, different situations have different irregularities, and 2) irregularities are scarce in videos. Therefore, we formulate video temporal irregularity detection as fake data detection via the discriminative framework of a designed 3D-GAN. This new formulation only employs regular videos during the training phase and detects irregularities according to the deviation estimated by the discriminator of 3D-GAN. We take regular videos as real data and construct a 3D-GAN to learn the distribution of regular videos during the training phase. Since testing data contain irregular videos or fake data, whose distribution is different from regular videos or real data, the trained discriminator of our networks is able to detect temporal regularities and irregularities. Experiments show that 3D-GANs outperforms 2D-GANs in temporal irregularity detection, and demonstrate the effectiveness and competitive performance of our approach on anomaly detection datasets.