TY - GEN
T1 - Waveear
T2 - 17th ACM International Conference on Mobile Systems, Applications, and Services, MobiSys 2019
AU - Xu, Chenhan
AU - Li, Zhengxiong
AU - Zhang, Hanbin
AU - Rathore, Aditya Singh
AU - Li, Huining
AU - Song, Chen
AU - Wang, Kun
AU - Xu, Wenyao
N1 - Publisher Copyright:
© 2019 Association for Computing Machinery.
PY - 2019/6/12
Y1 - 2019/6/12
N2 - Voice-user interface (VUI) has become an integral component in modern personal devices (e.g., smartphones, voice assistant) by fundamentally evolving the information sharing between the user and device. Acoustic sensing for VUI is designed to sense all acoustic objects; however, the existing VUI mechanism can only offer low-quality speech sensing. This is due to the audible and inaudible interference from complex ambient noise that limits the performance of VUI by causing denial-of-service (DoS) of user requests. Therefore, it is of paramount importance to enable noise-resistant speech sensing in VUI for executing critical tasks with superior efficiency and precision in robust environments. To this end, we investigate the feasibility of employing radio-frequency signals, such as millimeter wave (mmWave) for sensing the noise-resistant voice of an individual. We first perform an in-depth study behind the rationale of voice generation and resulting vocal vibrations. From the obtained insights, we present WaveEar, an end-to-end noise-resistant speech sensing system. WaveEar comprises a low-cost mmWave probe to localize the position of the speaker among multiple people and direct the mmWave signals towards the near-throat region of the speaker for sensing his/her vocal vibrations. The received signal, containing the speech information, is fed to our novel deep neural network for recovering the voice through exhaustive extraction. Our experimental evaluation under real-world scenarios with 21 participants shows the effectiveness of WaveEar to precisely infer the noise-resistant voice and enable a pervasive VUI in modern electronic devices.
AB - Voice-user interface (VUI) has become an integral component in modern personal devices (e.g., smartphones, voice assistant) by fundamentally evolving the information sharing between the user and device. Acoustic sensing for VUI is designed to sense all acoustic objects; however, the existing VUI mechanism can only offer low-quality speech sensing. This is due to the audible and inaudible interference from complex ambient noise that limits the performance of VUI by causing denial-of-service (DoS) of user requests. Therefore, it is of paramount importance to enable noise-resistant speech sensing in VUI for executing critical tasks with superior efficiency and precision in robust environments. To this end, we investigate the feasibility of employing radio-frequency signals, such as millimeter wave (mmWave) for sensing the noise-resistant voice of an individual. We first perform an in-depth study behind the rationale of voice generation and resulting vocal vibrations. From the obtained insights, we present WaveEar, an end-to-end noise-resistant speech sensing system. WaveEar comprises a low-cost mmWave probe to localize the position of the speaker among multiple people and direct the mmWave signals towards the near-throat region of the speaker for sensing his/her vocal vibrations. The received signal, containing the speech information, is fed to our novel deep neural network for recovering the voice through exhaustive extraction. Our experimental evaluation under real-world scenarios with 21 participants shows the effectiveness of WaveEar to precisely infer the noise-resistant voice and enable a pervasive VUI in modern electronic devices.
KW - MmWave
KW - Neural network
KW - Speech recognition
KW - Voice-user interface
UR - https://www.scopus.com/pages/publications/85069199963
U2 - 10.1145/3307334.3326073
DO - 10.1145/3307334.3326073
M3 - Conference contribution
AN - SCOPUS:85069199963
T3 - MobiSys 2019 - Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services
SP - 14
EP - 26
BT - MobiSys 2019 - Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services
PB - Association for Computing Machinery, Inc
Y2 - 17 June 2019 through 21 June 2019
ER -