Structural similarity for document image classification and retrieval

This paper presents a novel approach to defining document image structural similarity for the applications of classification and retrieval. We first build a codebook of SURF descriptors extracted from a set of representative training images. We then encode each document and model the spatial relationships between them by recursively partitioning the image and computing histograms of codewords in each partition. A random forest classifier is trained with the resulting features, and used for classification and retrieval. We demonstrate the effectiveness of our approach on table and tax form retrieval, and show that the proposed method outperforms previous approaches even when the training data is limited.