User guided semantic image adaptation for mobile display devices

This paper proposes a novel semantics-based consumer photo adaptation scheme for users of small-display mobile devices. The main contributions of the proposed scheme are: 1) seamless integration of mobile user supplied semantic information with low level image features to identify semantically important regions-of-interest (ROI), and 2) perceptually optimized adaptation for photo display on mobile devices. In order to bridge the semantic gap in photo search in a consumer photo collection and to perform appropriate adaptation of perceptually important region-of-interest from each photo to fit small displays on the mobile devices, we design a Bayesian fusion approach to properly integrate low level features with high level semantics. Low level features are extracted in a bottom-up fashion while the high level semantics is applied in a top-down style. Extensive experiments have been carried out based on several common events defined in the Kodak consumer photo database. These experiments show that by utilizing the semantics provided by the mobile device users, perceptually consistent adaptation can be effectively carried out. This new semantic adaptation scheme is able to outperform the conventional attention-based scheme for the common events and produce desired regions for the small displays on mobile devices.