Hierarchical sparse coding for visual co-occurrence discovery

In this chapter, we investigate soft assignments instead of hard assignments used in Chap. 2 and propose a hierarchical sparse coding method to learn representative mid-level visual phrases. Given multiple types of low-level visual primitive features, we first learn their sparse codes, respectively. Then, we cast these sparse codes into mid-level visual phrases by spatial pooling in spatial space. Besides that, we also concatenate the sparse codes of multiple feature types to discover feature phrases in feature space. After that, we further learn the sparse codes for the formed visual phrases in spatial and feature spaces, which can be more representative compared with the low-level sparse codes of visual primitive features. The superior results on various tasks of visual categorization and pattern discovery validate the effectiveness of the proposed approach.