Anomalous Pattern Recognition in Vital Health Signals via Multimodal Fusion

Increasingly, caregiving to senior citizens and patients requires monitoring of vital signs like heartbeat, respiration, and blood pressure for an extended period. In this paper, we propose a multimodal synchronized biological signal analysis using a deep neural network-based model that may learn to classify different anomalous patterns. The proposed cepstral-based peak fusion technique is designed to model the robust characterization of each biological signal by combining the list of dominant peaks in the input signal and its corresponding cepstrum. This works as an input to the following multimodal anomaly detection process that not only enables accurate identification and localization of aberrant signal patterns but also facilitates the proposed model to adopt an individual’s unique health characteristics over time. In this work, we use Electrocardiogram (ECG), Femoral Pulse, Photoplethysmogram (PPG), and Body Temperature to monitor an individual’s health condition. In both publicly available datasets as well as our lab-based study with 10 participants, the proposed cepstral-based fusion module attains around 7 to 10 % improvement over the baseline of time-domain analysis and the proposed deep learning classifier reports an average accuracy of 95.5 % with 8 classes and 93 % (improvement of 3 % ) with 17 classes.