Takaki Makino

Tuning support vector machines for biomedical named entity recognition

We explore the use of Support Vector Machines (SVMs) for biomedical named entity recognition. To make the SVM training with the available largest corpus - the GENIA corpus - tractable, we propose to split the non-entity class into sub-classes, using part-of-speech information. In addition, we explore new features such as word cache and the states of an HMM trained by unsupervised learning. Experiments on the GENIA corpus show that our class splitting technique not only enables the training with the GENIA corpus but also improves the accuracy. The proposed new features also contribute to improve the accuracy. We compare our SVM-based recognition system with a system using Maximum Entropy tagging method.

LiLFeS - Towards a Practical HPSG Parser

This paper presents the LiLFeS system, an efficient feature-structure description language for HPSG. The core engine of LiLFeS is an Abstract Machine for Attribute-Value Logics, proposed by Carpenter and Qu. Basic design policies, the current status, and performance evaluation of the LiLFeS system are described. The paper discusses two implementations of the LiLFeS. The first one is based on an emulator of the abstract machine, while the second one uses a native-code compiler and therefore is much more efficient than the first one.

The LiLFeS Abstract Machine and its evaluation with the LinGO grammar

This article evaluates the efficiency of the LiLFeS abstract machine by performing parsing tasks with the LinGO English resource grammar. The instruction set of the abstract machine is optimized for efficient processing of definite clause programs and typed feature structures. LiLFeS also supports various tools required for efficient parsing (e.g. efficient copying, a built-in CFG parser) and the constructions of standard Prolog (e.g. cut, assertions, negation as failure). Several parsers and large-scale grammars, including the LinGO grammar, have been implemented in or ported to LiLFeS. Precise empirical results with the LinGO grammar are provided to allow comparison with other systems. The experimental results demonstrate the efficiency of the LiLFeS abstract machine.

On-line discovery of temporal-difference networks

We present an algorithm for on-line, incremental discovery of temporal-difference (TD) networks. The key contribution is the establishment of three criteria to expand a node in TD network: a node is expanded when the node is well-known, independent, and has a prediction error that requires further explanation. Since none of these criteria requires centralized calculation operations, they are easily computed in a parallel and distributed manner, and scalable for bigger problems compared to other discovery methods of predictive state representations. Through computer experiments, we demonstrate the empirical effectiveness of our algorithm.

An indexing scheme for typed feature structures

This paper describes an indexing substrate for typed feature structures (ISTFS), which is an efficient retrieval engine for typed feature structures. Given a set of typed feature structures, the ISTFS efficiently retrieves its subset whose elements are unifiable or in a subsumption relation with a query feature structure. The efficiency of the ISTFS is achieved by calculating a unifiability checking table prior to retrieval and finding the best index paths dynamically.

Proto-predictive representation of states with simple recurrent temporal-difference networks

We propose a new neural network architecture, called Simple Recurrent Temporal-Difference Networks (SR-TDNs), that learns to predict future observations in partially observable environments. SR-TDNs incorporate the structure of simple recurrent neural networks (SRNs) into temporal-difference (TD) networks to use proto-predictive representation of states. Although they deviate from the principle of predictive representations to ground state representations on observations, they follow the same learning strategy as TD networks, i.e., applying TD-learning to general predictions. Simulation experiments revealed that SR-TDNs can correctly represent states with an incomplete set of core tests (question networks), and consequently, SR-TDNs have better on-line learning capacity than TD networks in various environments.

Pseudo-Orthogonalization of Memory Patterns for Associative Memory

A new method for improving the storage capacity of associative memory models on a neural network is proposed. The storage capacity of the network increases in proportion to the network size in the case of random patterns, but, in general, the capacity suffers from correlation among memory patterns. Numerous solutions to this problem have been proposed so far, but their high computational cost limits their scalability. In this paper, we propose a novel and simple solution that is locally computable without any iteration. Our method involves XNOR masking of the original memory patterns with random patterns, and the masked patterns and masks are concatenated. The resulting decorrelated patterns allow higher storage capacity at the cost of the pattern length. Furthermore, the increase in the pattern length can be reduced through blockwise masking, which results in a small amount of capacity loss. Movie replay and image recognition are presented as examples to demonstrate the scalability of the proposed method.

Bridging the Knowledge Gap between Research and Education through Textbooks

We describe a web-based system that helps undergraduate students learn about research at their university through the medium of textbooks. The system, called University on Textbooks, links textbooks to research articles written by university researchers, enabling people to navigate between introductory materials in textbooks and specific topics reported in research articles. Bidirectional links are generated from semantic statements created manually for topical passages from textbooks and abstracts of articles. The semantic statements are created using an ontology that is based on a description logic, so a reasoner can infer the semantic similarity between pairs of statements. To demonstrate the feasibility and effectiveness of our approach, we present a prototype containing 100 semantic statements describing passages from an undergraduate life sciences textbook in Japanese that have been linked to semantic statements created for 392 research articles from life sciences.

Literature-Based Knowledge Discovery from Relationship Associations Based on a DL Ontology Created from MeSH

Literature-based knowledge discovery generates potential discoveries from associations between specific concepts that have been previously reported in the literature. However, because the associations are generally between individual concepts, the knowledge of specific relationships between those concepts is lost. A description logic (DL) ontology adds a set of logically defined relationship types, called properties, to a classification of concepts for a particular knowledge domain. Properties can represent specific relationships between instances of concepts used to describe the things studied by a particular researcher. These relationships form a “triple” consisting of a domain instance, a range instance, and the property specifying the way those instances are related. A “relationship association” is a pair of relationship triples where one of the instances from each relationship can be determined to be semantically equivalent. In this paper, we report our work to structure a subset of more than 1300 terms from the Medical Subject Headings (MeSH) controlled vocabulary into a DL ontology, and to use that DL ontology to create a corpus of A-Boxes, which we call “semantic statements”, each of which describes one of 392 research articles that we selected from MEDLINE. Relationship associations were extracted from the corpus of semantic statements using a previously reported technique. Then, by making the assumption of the transitivity of association used in literature-based knowledge discovery, we generate hypothetical relationship associations by combining pairs of relationship associations. We then evaluate the “interestingness” of those candidate knowledge discoveries from a life science perspective.