Mining GPS traces and visual words for event classification

It is of great interest to recognize semantic events (e.g., hiking, skiing, party), in particular when given a collection of personal photos, where each photo is tagged with a timestamp and GPS (Global Positioning System) information at the capture. We address this emerging multiclass classification problem by mining informative features derived from traces of GPS coordinates and a bag of visual words, both based on the entire collection as opposed to individual photos. Considering that semantic events are best characterized by a compositional description of the visual content in terms of the co-occurrence of objects and scenes, we focus on mining compositional features (equivalent to word combinations in the "bag-of-words" method) that have better discriminative and descriptive abilities than individual features. In order to handle the combinatorial complexity in discovering such compositional features, we apply a data mining method based on frequent itemset mining (FIM). Complementary features are also derived from GPS traces and mined to characterize the underlying movement patterns of various event types. Upon compositional feature mining, we perform multiclass AdaBoost to solve the multiclass problem. Based on a dataset of eight event classes and a total of more than 3000 geotagged images from 88 events, experimental results using leave-one-out cross validation have shown the synergy of all of the components in our proposed approach to event classification.