Suncong Zheng

Self-Taught convolutional neural networks for short text clustering

Short text clustering is a challenging problem due to its sparseness of text representation. Here we propose a flexible Self-Taught Convolutional neural network framework for Short Text Clustering (dubbed STC2), which can flexibly and successfully incorporate more useful semantic features and learn non-biased deep text representation in an unsupervised manner. In our framework, the original raw text features are firstly embedded into compact binary codes by using one existing unsupervised dimensionality reduction method. Then, word embeddings are explored and fed into convolutional neural networks to learn deep feature representations, meanwhile the output units are used to fit the pre-trained binary codes in the training process. Finally, we get the optimal clusters by employing K-means to cluster the learned representations. Extensive experimental results demonstrate that the proposed framework is effective, flexible and outperform several popular clustering methods when tested on three public short text datasets.

A neural network framework for relation extraction

Relation extraction is to identify the relationship of two given entities in the text. It is an important step in the task of knowledge extraction. Most conventional methods for the task of relation extraction focus on designing effective handcrafted features or learning a semantic representation of the whole sentence. Sentences with the same relationship always share the similar expressions. Besides, the semantic properties of given entities can also help to distinguish some confusing relations. Based on the above observations, we propose a neural network based framework for relation classification. It can simultaneously learn the relation patterns information and the semantic properties of given entities. In this framework, we explore two specific models: the CNN-based model and LSTM-based model. We conduct experiments on two public datasets: the SemEval-2010 Task8 dataset and the ACE05 dataset. The proposed method achieves the state-of-the-art result without using any external information. Additionally, the experimental results also show that our approach can represent the semantic relationship of the given entities effectively.

Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme.

Joint extraction of entities and relations is an important task in information extraction. To tackle this problem, we firstly propose a novel tagging scheme that can convert the joint extraction task to a tagging problem. Then, based on our tagging scheme, we study different end-to-end models to extract entities and their relations directly, without identifying entities and relations separately. We conduct experiments on a public dataset produced by distant supervision method and the experimental results show that the tagging based methods are better than most of the existing pipelined and joint learning methods. Whats more, the end-to-end model proposed in this paper, achieves the best results on the public dataset.

Joint Extraction of Multiple Relations and Entities by Using a Hybrid Neural Network

This paper proposes a novel end-to-end neural model to jointly extract entities and relations in a sentence. Unlike most existing approaches, the proposed model uses a hybrid neural network to automatically learn sentence features and does not rely on any Natural Language Processing (NLP) tools, such as dependency parser. Our model is further capable of modeling multiple relations and their corresponding entity pairs simultaneously. Experiments on the CoNLL04 dataset demonstrate that our model using only word embeddings as input features achieves state-of-the-art performance.

A Bidirectional Hierarchical Skip-Gram model for text topic embedding

Taking advantage of the large scale corpus on the web to effectively and efficiently mine the topics within texts is an essential problem in the era of big data. We focus on the problem of learning text topic embedding in an unsupervised manner, which enjoys the properties of efficiency and scalability. Text topic embedding represents words and documents in a semantic topic space, in which the words and documents with similar topic will be embedded close to each other. When compared with conventional topic models, which implicitly capture the document-level word co-occurrence patterns, text topic embedding alleviates the data sparsity problem and captures the semantic relevance between different words and documents. To model text topic embedding, we propose a Bidirectional Hierarchical Skip-Gram model (BHSG) based on skip-gram model. BHSG includes two components: semantic generation module to learn semantic relevance between texts and topic enhance module to produce the text topic embedding based on text embedding learned in the former module. We evaluated our method on two kinds of topic-related tasks: text classification and information retrieval. The experimental results on four public datasets and one dataset we provide all demonstrate that our proposed method can achieve a better performance.

Ensemble of Feature Sets and Classification Methods for Stance Detection

Stance detection is the task of automatically determining the author’s favorability towards a given target. However, the target may not be explicitly mentioned in the text and even someone may refer some positive opinions to against the target, which make the task more difficult. In this paper, we describe an ensemble framework which integrates various feature sets and classification methods, and does not consist any handcrafted templates or rules to help stance detection. We submit our solution to NLPCC 2016 shared task: Detecting Stance in Chinese Weibo (Task A), which is a supervised task towards five targets. The official results show that our solution of the team “CBrain” achieves one 1st place and one 2nd place on these targets, and the overall ranking is 4th out of 16 teams. Our code is available at https://github.com/jacoxu/2016NLPCC_Stance_Detection.

A Novel Hierarchical Convolutional Neural Network for Question Answering over Paragraphs

The question of classical Factoid Question Answering (FQA) task is always in the form of a single sentence. There also exists another kind of FQA task, whose question is a descriptive paragraph, such as quiz bowl question answering. Recently, some works try to automatically answer paragraph questions by applying machine learning methods. However, these methods neglect the correlation information between sentences in a paragraph and do not take full advantage of answer embedding information. In this paper, we propose a novel Hierarchical Convolutional Neural Network, called HCNN-E, to settle the task by considering ordinal information of sentences in paragraph and the information of answer embeddings. The experimental results on two public datasets demonstrate the effectiveness of proposed method, and the proposed method can achieve approximately 10% - 20% improvements, when comparing with the baselines.