Integrating recognition and reasoning in smart environments

Our goal is to develop 'smart indoor environments' that are monitored unobtrusively by biometric capture devices, such as video cameras, microphones, etc. Such environments will keep track of their occupants and be capable of answering queries about the occupants' whereabouts. In order to develop a unified model that is applicable across diverse biometric modalities, we propose an abstract state transition framework in which different recognition steps are abstracted by events, and the reasoning necessary to effect state transitions is abstracted by a transition function. We define the metrics of 'precision' and 'recall' of a smart environment to evaluate how well it tracks its occupants. We show how the overall performance of our smart environment can be improved through the use of spatio-temporal knowledge of the environment. A prototype based upon our proposed abstract framework indicates that integrating recognition and reasoning capabilities substantially improves the overall performance of the environment.