TY - GEN
T1 - Ig3D
T2 - 18th European Conference on Computer Vision, ECCV 2024
AU - Dong, Lu
AU - Wang, Xiao
AU - Setlur, Srirangaraj
AU - Govindaraju, Venu
AU - Nwogu, Ifeoma
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
PY - 2025
Y1 - 2025
N2 - Reconstructing 3D faces with facial geometry from single images has allowed for major advances in animation, generative models, and virtual reality. However, this ability to represent faces with their 3D features is not as fully explored by the facial expression inference (FEI) community. This study therefore aims to investigate the impacts of integrating such 3D representations into the FEI task, specifically for facial expression classification and face-based valence-arousal (VA) estimation. To achieve this, we first evaluate the performance of two 3D face representations (both based on the 3D morphable model, FLAME) for the FEI tasks. We further explore two fusion architectures, intermediate fusion, and late fusion, for integrating the 3D face representations with existing 2D inference frameworks. To evaluate the proposed architecture, we extract the corresponding 3D representations and perform extensive experiments on the AffectNet and RAF-DB datasets. The experimental results show that our method outperforms the state-of-the-art in AffectNet VA estimation and RAF-DB classification tasks. Furthermore, our method can serve as a complement to other existing methods to boost performance in many emotion inference tasks.
AB - Reconstructing 3D faces with facial geometry from single images has allowed for major advances in animation, generative models, and virtual reality. However, this ability to represent faces with their 3D features is not as fully explored by the facial expression inference (FEI) community. This study therefore aims to investigate the impacts of integrating such 3D representations into the FEI task, specifically for facial expression classification and face-based valence-arousal (VA) estimation. To achieve this, we first evaluate the performance of two 3D face representations (both based on the 3D morphable model, FLAME) for the FEI tasks. We further explore two fusion architectures, intermediate fusion, and late fusion, for integrating the 3D face representations with existing 2D inference frameworks. To evaluate the proposed architecture, we extract the corresponding 3D representations and perform extensive experiments on the AffectNet and RAF-DB datasets. The experimental results show that our method outperforms the state-of-the-art in AffectNet VA estimation and RAF-DB classification tasks. Furthermore, our method can serve as a complement to other existing methods to boost performance in many emotion inference tasks.
KW - 3D Face Representations
KW - Facial Expression Inference
KW - Intermediate and Late Fusion
UR - https://www.scopus.com/pages/publications/105014495226
U2 - 10.1007/978-3-031-91581-9_29
DO - 10.1007/978-3-031-91581-9_29
M3 - Conference contribution
AN - SCOPUS:105014495226
SN - 9783031915802
T3 - Lecture Notes in Computer Science
SP - 404
EP - 421
BT - Computer Vision – ECCV 2024 Workshops , Proceedings
A2 - Del Bue, Alessio
A2 - Canton, Cristian
A2 - Pont-Tuset, Jordi
A2 - Tommasi, Tatiana
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 29 September 2024 through 4 October 2024
ER -