Simultaneous clustering and tracklet linking for multi-face tracking in videos

We describe a novel method that simultaneously clusters and associates short sequences of detected faces (termed as face track lets) in videos. The rationale of our method is that face track let clustering and linking are related problems that can benefit from the solutions of each other. Our method is based on a hidden Markov random field model that represents the joint dependencies of cluster labels and track let linking associations. We provide an efficient algorithm based on constrained clustering and optimal matching for the simultaneous inference of cluster labels and track let associations. We demonstrate significant improvements on the state-of-the-art results in face tracking and clustering performances on several video datasets.