Long-term memory networks for question answering

Question answering is an important and difficult task in the natural language process-ing domain, because many basic natural lan-guage processing tasks can be cast into a ques-tion answering task. Several deep neural net-work architectures have been developed re-cently, which employ memory and inference components to memorize and reason over text information, and generate answers to ques-tions. However, a major drawback of many such models is that they are capable of only generating single-word answers. In addition, they require large amount of training data to generate accurate answers. In this paper, we introduce the Long-Term Memory Network (LTMN), which incorporates both an exter-nal memory module and a Long Short-Term Memory (LSTM) module to comprehend the input data and generate multi-word answers. The LTMN model can be trained end-to-end using back-propagation and requires minimal supervision. We test our model on two syn-thetic data sets (based on Facebook's bAbI data set) and the real-world Stanford ques-tion answering data set, and show that it can achieve state-of-the-art performance.