Huang Changning

CSeg&Tagl.0: A Practical Word Segmenter and POS Tagger for Chinese Texts

Chinese word segmentation and POS tagging are two key techniques in many applications in Chinese information processing. Great efforts have been paid to the research in the last decade, but unfortunately, no practical system with high performance for unrestricted texts is available up to date. CSeg&Tag1.0, a Chinese word segmenter and POS tagger which unifies these two procedures into one model, is introduced in this paper. The preliminary open tests show that the segmentation precision of CSeg&Tag1.0 is about 98.0% - 99.3%, POS tagging precision about 91.0% - 97.1%, and the recall and precision for unknown words are ranging from 95.0% to 99.0% and from 87.6% to 95.3% respectively. The processing speed is about 100 characters per second on Pentium 133 PC. The work of improving the performance of the system is still ongoing.

A Quasi-Dependency Model for the Structural Analysis of Chinese BaseNPs

The paper puts forward a quasi-dependency model for structural analysis of Chinese baseNPs and a MDL-based algorithm for quasi-dependency-strength acquisition. The experiments show that the proposed model is more suitable for Chinese baseNP analysis and the proposed MDL-based algorithm is superior to the traditional ML-based algorithm. The paper also discusses the problem of incorporating the linguistic knowledge into the above statistical model.

Knowledge acquisition and Chinese parsing based on corpus

In Natural Language Processing (NLP), one key problem is how to design a robust and effective parsing system. In this paper, we will introduce a corpus- based Chinese parsing system. Our efforts are concetrated on: (1) knowledge acquisition and representation; and (2) the parsing scheme. The knowledge of this system is principally extracted from analyzed corpus, others are a few grammatical principles, i.e. the four axioms of the Dependency Grammar (DG). In addition, we also propose the fifth axiom of DG to support the parsing of Chinese sentences.

Automatic error detection and correction approach in Chinese text based on features and learning

Language models adopted by most existing error detection and correction approaches of Chinese text are N-Gram models of characters, words or POS tags. Their deficiencies are that only the local language constraint is employed and there is no language model unification process. A feature-based automatic error detection and correction approach is presented. It uses both local language features and wide-scope semantic features. Winnow is adopted in the learning step. In experiment, this method achieved an error detection recall rate of 85%, precise rate of 41% and error correction rate of 51%. It shows that the approach performs better than the existing approaches based on N-Gram models.

Adding New Words into A Chinese Thesaurus

In this paper, we study the problem of adding a large number of new words into a Chinese thesaurus according to their definitions in a Chinese dictionary, while minimizing the effort of hand tagging. To deal with the problem, we first make use of a kind of supervised learning technique to learn a set of defining formats for each class in the thesaurus, which tries to characterize the regularities about the definitions of the words in the class. We then use traditional techniques in Graph theory to derive a minimal subset of the new words to be added into the thesaurus, which meets the following condition: if we add the new words in the subset into the thesaurus by hand, the other new words can be added into the thesaurus automatically by matching their definitions with the defining formats of each class in the thesaurus. The method uses little, if any, language-specific or thesaurus-specific knowledge, and can be applied to the thesauri of other languages.In this paper, we study the problem of adding a large number of new words into a Chinese thesaurus according to their definitions in a Chinese dictionary, while minimizing the effort of hand tagging. To deal with the problem, we first make use of a kind of supervised learning technique to learn a set of defining formats for each class in the thesaurus, which tries to characterize the regularities about the definitions of the words in the class. We then use traditional techniques in Graph theory to derive a minimal subset of the new words to be added into the thesaurus, which meets the following condition: if we add the new words in the subset into the thesaurus by hand, the other new words can be added into the thesaurus automatically by matching their definitions with the defining formats of each class in the thesaurus. The method uses little, if any, language-specific or thesaurus-specific knowledge, and can be applied to the thesauri of other languages.