III: Small: Purposeful Conversational Agents based on Hierarchical Knowledge Graphs

There has been tremendous progress recently in the area of open domain conversational agents (CA), also referred to as chatbots or socialbots, by leveraging neural response generators and large, diverse training corpora. The recent Alexa Prize competition resulted in socialbots that can engage in prolonged conversations on any topic, paving the way for meaningful use. These systems are limited to chitchat, where the conversation meanders from topic to topic without any specific goal. There is now an opportunity (and need) to develop models for purposeful conversations, motivated by societal goals. This facilitates compelling solutions such as chatbots to assist with mental health issues, providing companionship to senior citizens, and even combating disinformation. CAs can be configured for different personalities, diverse and multilingual populations and thus hold great potential to address societal problems at scale. Realizing this goal requires advances on multiple fronts, including ethical issues and trustworthiness. This project addresses two key technical challenges. First, the project eliminates artifacts of neural response generation such as inconsistency, incoherence, and repetition. These responses diminish the trustworthiness of socialbots and hence their impact. Second, the project addresses the computational frameworks for purposeful conversations, specifically, persuasion. This project will result in a fully functional CA, that incorporates factual knowledge, logical reasoning, and conversational strategies found in human-to-human dialogue, to engage in coherent, interesting and persuasive conversations across several topics related to societal issues. Purposeful conversational agents must engage in meaningful exchanges with a motive: a persuasive conversation may cycle through inquiry, negotiation, or deliberation interspersed with chit-chat and anecdotes. In order to realize these objectives, this project incorporates the use of specially constructed multi-layered knowledge graphs, along with state-of-the-art neural response generators, intent classifiers, and a robust natural language understanding and dialogue management module. The core layers of the open knowledge network consist of consolidated concepts and relationships from existing open knowledge networks and causal graphs, suitably enhanced to facilitate neuro-symbolic approaches for knowledge grounded conversation. The peripheral layers constitute argumentation graphs which are automatically constructed through processing existing corpora of persuasive conversations and extracting elemental discourse units and relationships between them. An innovative neuro-symbolic approach is necessary to (i) avoid the typical “hallucinations” exhibited by neural response generators alone and (ii) intelligently navigate through the various stages of a persuasive conversation. The neural response generators are configured to efficiently retrieve, reason and incorporate information from the open knowledge network, in order to generate factually correct, coherent, engaging and persuasive responses. Both novel and traditional metrics will be used in order to assess the quality of the generated conversations and measure its impact. Automated evaluation will be conducted on representative data sets; the prototypes will also be evaluated by experts who will provide feedback regarding viability for future deployment. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.