Data-Driven Quickest Change Detection in (Hidden) Markov Models

The paper investigates the problems of quickest change detection in Markov models and hidden Markov models (HMMs). Sequential observations are taken from a (hidden) Markov model. At some unknown time, an event occurs in the system and changes the transition kernel of the Markov model and/or the emission probability of the HMM. The objective is to detect the change quickly while controlling the average running length (ARL) to false alarm. The data-driven setting is studied, where no knowledge of the pre- or post-change distributions is available. Kernel-based data-driven algorithms are developed, which can be applied in the setting with continuous state, can be updated in a recursive fashion, and are computationally efficient. Lower bounds on the ARL and upper bounds on the worst-case average detection delay (WADD) are derived. The WADD is at most of the order of the logarithm of the ARL. The algorithms are further numerically validated on two practical problems of fault detection in DC microgrid and photovoltaic systems.