A joint visual-inertial image registration for mobile HDR imaging

Robust image alignment is a necessary and challenging step for numerous computational photography applications. In particular, large camera motion poses significant challenge to Mobile High Dynamic Range (HDR) Imaging due to hand-held capture of input images and limited computational resources. Aligning images only by detecting and matching image features is computationally expensive and can also be erratic. We present a robust multi-sensory method for aligning exposure bracketed images on mobile cameras. We use inertial sensor based camera pose estimate to pre-warp images and iteratively align them by minimizing alignment error. We also simultaneously estimate local motion masks which can be used to eliminate ghosting artifacts in the final HDR image. We collected HDR image dataset with diverse scenes along with inertial sensor data, which is a novel contribution and have evaluated our performance with existing mobile HDR image alignment techniques in literature.