TY - GEN
T1 - IMUFace
T2 - 23rd ACM Conference on Embedded Networked Sensor Systems, SenSys 2025
AU - Yao, Xianrong
AU - Yu, Chengzhang
AU - Hu, Lingde
AU - Jin, Yincheng
AU - Gao, Yang
AU - Jin, Zhanpeng
N1 - Publisher Copyright:
© 2025 Copyright held by the owner/author(s).
PY - 2025/5/6
Y1 - 2025/5/6
N2 - Facial expressions are vital for effective communication, conveying emotions and health status. Traditional analysis methods, like manual annotations and geometric models, are labor-intensive and inadequate for complex situations. While vision-based approaches improve accuracy, they often struggle with environmental constraints and privacy concerns. Non-visual wearables offer flexibility but can be uncomfortable and power-hungry. To overcome these issues, we introduce IMUFace, an innovative earplug platform that uses inertial measurement units (IMUs) for real-time facial expression reconstruction. IMUFace captures facial motion data through IMUs in headphones and processes it with a deep learning model to estimate facial landmarks accurately. These predictions are then fitted to the FLAME model, creating realistic 3D facial animations. Compact and low-power, IMUFace represents a significant advancement in generating 3D facial animations for everyday use.
AB - Facial expressions are vital for effective communication, conveying emotions and health status. Traditional analysis methods, like manual annotations and geometric models, are labor-intensive and inadequate for complex situations. While vision-based approaches improve accuracy, they often struggle with environmental constraints and privacy concerns. Non-visual wearables offer flexibility but can be uncomfortable and power-hungry. To overcome these issues, we introduce IMUFace, an innovative earplug platform that uses inertial measurement units (IMUs) for real-time facial expression reconstruction. IMUFace captures facial motion data through IMUs in headphones and processes it with a deep learning model to estimate facial landmarks accurately. These predictions are then fitted to the FLAME model, creating realistic 3D facial animations. Compact and low-power, IMUFace represents a significant advancement in generating 3D facial animations for everyday use.
KW - 3D facial reconstruction
KW - mobile computing
KW - singleear biosensing
KW - wearable sensing
UR - https://www.scopus.com/pages/publications/105035723538
U2 - 10.1145/3715014.3724034
DO - 10.1145/3715014.3724034
M3 - Conference contribution
AN - SCOPUS:105035723538
T3 - ACM SenSys 2025 - 23rd ACM Conference on Embedded Networked Sensor Systems, In Transactions to Conference Embedded Artificial Intelligence and Sensing Systems
SP - 614
EP - 615
BT - ACM SenSys 2025 - 23rd ACM Conference on Embedded Networked Sensor Systems, In Transactions to Conference Embedded Artificial Intelligence and Sensing Systems
PB - Association for Computing Machinery, Inc
Y2 - 6 May 2025 through 9 May 2025
ER -