Knowledge-Aware LSTM for Machine Comprehension

Abstract

Machine Comprehension (MC) of text is the problem to answer a query based on a given document. Although MC has been very popular recently, it still have some serious weaknesses which rely only on query-to-document interaction or its learning is just heavily dependent on the training data. To take advantage of external knowledge to improve neural networks for MC, we propose a novel knowledge enhanced recurrent neural model, called knowledge-aware LSTM (k-LSTM), an extension to basic LSTM cells, designed to exploit external knowledge bases (KBs) to improve neural networks for MC task. To incorporate KBs with contextual information effectively from the currently text, k-LSTM employs an compositional attention mechanism to adaptively decide whether to attend to KBs and which information from external knowledge is useful. Furthermore, we present our knowledge enhanced neural network, called Knowledge-guided DIM Reader (K-DIM Reader), which is a novel knowledge-aware compositional attention neural network architecture, employing the k-LSTM in our framework. By stringing external background knowledge together and imposing compositional attention interaction that regulate their interaction, K-DIM Reader effectively learns to perform reading comprehension processes that are directly inferred from the data in an end-to-end approach. We show our proposed models strength, robustness and interpretability on the challenging MC datasets, achieving significant improvements on SQuAD dataset [1] and obtaining new state-of-the-art results on both Cloze-style datasets, CBTest [2] and CNN news [3]. In particular, we further extend 6 popular end-to-end neural MC models using k-LSTM incorporating knowledge into models for improving MC, and evaluate their performance on both well-known MC datasets. We demonstrate that neural model with external knowledge improves performance on MC task.