The Role of Context in Prosocial Affect Recognition

Although prosocial affect, defined as emotional responses that signal care, concern, or shared emotional alignment with others, plays a central role in social interaction, it remains underexplored in affective computing. Existing work has largely equated prosocial behavior with narrowly defined empathy, often measured through facial expressions of sadness or distress in response to a predefined cue. These works implicitly assume (i) a distinctive facial signature and (ii) a single, unitary affective state. Recognizing that the space of prosocial affective states spans more than just empathetic distress, we ask: (a) to what extent can general prosocial affect be inferred from a single person's face alone, and (b) how does social context shape this recognition process? Hence, to explore these questions, we annotate prosocial moments in a television drama series, linking each instance to dialog, surrounding situation, and onscreen evidence; and then evaluate models under increasingly expressive assumptions. A supervised ResNet-50 trained on face crops does not generalize, so we reformulate prosocial recognition as an inlier verification problem and train a oneclass SVM on the same facial embeddings. This improves stability but remains limited without context. We then introduce a context-grounded verification framework that integrates facial affect with conversational and situational cues derived from dialog, learning a distribution from positive examples and rejecting non-prosocial behavior as out-of-distribution. Two approaches to face-only verification yield 40% and 67% F1 scores, while context latents achieve 9 4% F1 for multimodal verification and 93% for dialog-only held-out generalization. Overall, we observe that prosocial affect is better characterized as a context-dependent socioemotional phenomenon than as a single individual's face-only visual signature.