TY - GEN
T1 - Toward Scalable ASL Education
T2 - 2026 CHI Conference on Human Factors in Computing Systems, CHI 2026
AU - Cai, Yongxiang
AU - Li, Zhenghao
AU - Lu, Taiting
AU - Zhu, Yanjun
AU - Wu, Yi Shan
AU - Zhang, Qingsen
AU - Xu, Xuhai
AU - Jin, Zhanpeng
AU - Gowda, Mahanth
AU - Jin, Yincheng
N1 - Publisher Copyright:
© 2026 Copyright held by the owner/author(s).
PY - 2026/4/13
Y1 - 2026/4/13
N2 - American Sign Language (ASL) is the primary language of many Deaf and Hard of Hearing (DHH) individuals. However, existing learning resources often lack timely, individualized feedback, leaving learners uncertain about signing accuracy. We introduce a novel egocentric ASL learning system that integrates stereo vision, error detection across four manual ASL parameters (handshape, orientation, location, movement), and large language model (LLM)-driven natural language feedback. To our knowledge, this is the first system to deliver error-aware, pedagogically grounded feedback for ASL learners. A formative study with 15 ASL teachers and 30 learners (both Deaf and hearing backgrounds) supports the motivation and design goals, while a system evaluation with 13 Deaf ASL participants (novice to advanced) practicing 230 signs provides initial evidence of system feasibility and short-term, pedagogically promising behavior within the primary user community. Across two complementary studies, we identify key design principles: prioritizing reliability over sensitivity, stratifying feedback by error severity, and leveraging egocentric alignment for natural practice. Collectively, these contributions establish a foundation for scalable ASL education and provide generalizable insights for designing AI-mediated feedback in Human-Computer Interaction (HCI).
AB - American Sign Language (ASL) is the primary language of many Deaf and Hard of Hearing (DHH) individuals. However, existing learning resources often lack timely, individualized feedback, leaving learners uncertain about signing accuracy. We introduce a novel egocentric ASL learning system that integrates stereo vision, error detection across four manual ASL parameters (handshape, orientation, location, movement), and large language model (LLM)-driven natural language feedback. To our knowledge, this is the first system to deliver error-aware, pedagogically grounded feedback for ASL learners. A formative study with 15 ASL teachers and 30 learners (both Deaf and hearing backgrounds) supports the motivation and design goals, while a system evaluation with 13 Deaf ASL participants (novice to advanced) practicing 230 signs provides initial evidence of system feasibility and short-term, pedagogically promising behavior within the primary user community. Across two complementary studies, we identify key design principles: prioritizing reliability over sensitivity, stratifying feedback by error severity, and leveraging egocentric alignment for natural practice. Collectively, these contributions establish a foundation for scalable ASL education and provide generalizable insights for designing AI-mediated feedback in Human-Computer Interaction (HCI).
KW - ASL Learning
KW - Error Detection
KW - Feedback
KW - Large Language Model (LLM)
UR - https://www.scopus.com/pages/publications/105038745437
U2 - 10.1145/3772318.3790774
DO - 10.1145/3772318.3790774
M3 - Conference contribution
AN - SCOPUS:105038745437
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - CHI 2026 - Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
A2 - Oliver, Nuria
A2 - Shamma, David A.
A2 - Candello, Heloisa
A2 - Cesar, Pablo
A2 - Lopes, Pedro
A2 - Bozzon, Alessandro
A2 - Kosch, Thomas
A2 - Liao, Vera
A2 - Ma, Xiaojuan
A2 - Artizzu, Valentino
A2 - Draxler, Fiona
A2 - Lopez, Gustavo
A2 - Reinschluessel, Anke V.
A2 - Tong, Xin
A2 - Toups Dugas, Phoebe O.
PB - Association for Computing Machinery
Y2 - 13 April 2026 through 17 April 2026
ER -