Haruhiko Takeuchi

Greedy information acquisition algorithm: a new information theoretic approach to dynamic information acquisition in neural networks

In this paper, we proopose a new information theoretic approach to competitive learning. The new approach is called greedy information acquisition , because networks try to absorb as much information as possible in every stage of learning. In the first phase, with minimum network architecture for realizing competition, information is maximized. In the second phase, a new unit is added, and thereby information is again increased as much as possible. This proceess continues until no more increase in information is possible. Through greedy information maximization, different sets of important features in input patterns can be cumulatively discovered in successive stages. We applied our approach to three problems: a dipole problem; a language classification problem; and a phonological feature detection problem. Experimental results confirmed that information maximization can be repeatedly applied and that different features in input patterns are gradually discovered. We also compared our method with conventional competitive learning and multivariate analysis. The experimental results confirmed that our new method can detect salient features in input patterns more clearly than the other methods.

A Quantitative Method for Analyzing Scan Path Data Obtained by Eye Tracker

Scan path is one of the most important metrics measured by eye tracking systems. This paper describes a new method for analyzing scan-path data based on the string-edit method that is popular for correcting human errors made at the input stage. We defined several cost functions for the substitution costs in the string-edit method, and applied the method to the scan-path data we had collected in a series of experiments for studying Web browsing behavior. We demonstrate the usefulness of our method and discuss the appropriate cost functions for the eye-tracking data.

The influence of web browsing experience on web-viewing behavior

The World Wide Web has become an important source of information, as much as traditional media like books, newspapers, and television. While there have been many studies on Web searching, research into Web-viewing behavior using eye-tracking systems has only recently begun [Pan et al., 2004]. Josephson and Holmes [2002] studied Web-viewing behavior focusing on the category of Web page visual design. They suggested that eye movements were affected by the following two factors: (1) visual design of Web pages and (2) habitually preferred path across the visual stimuli. However, these previous studies did not sufficiently consider the users experience. The purpose of this study is to investigate how past Web-browsing experience influences Web-viewing behavior. We used a detailed questionnaire to measure a users Web-browsing experience and analyzed the eye-tracking data based on the users prior Web experience.

Scan-path analysis by the string-edit method considering fixation duration

Dynamic aspects of eye-tracking data are important but difficult to analyze. With string based approaches, a sequence of fixations is analyzed, however, fixation duration is not addressed. Cristino et al. recently proposed to re-code a scan-path with a long fixation by repeating the code. The modified scan path includes both fixation duration and sequence of fixations. In studying multiple records by the string-edit method enhanced with cost functions, we compared the performance of the modified coding against the ordinary one. Furthermore, we derived representative scan paths to examine the distance among the web pages used as stimuli. The usefulness of our approach is demonstrated.

Modular structure generation by greedy network-growing algorithm

In this paper, we propose a new method to generate modular structures. In the method, the number of elements, that is, the number of competitive units is gradually increased. To control a process of module generation, we introduce two kinds of information, that is, unit and modular information. Unit information represents information content obtained by individual elements in all modules. On the other hand, modular information is information content obtained by each module. We try to increase both types of information simultaneously. We applied our method to two classification problems: random data classification and Web data classification. In both cases, we observed that modular structures were automatically generated.

Using psychological word database in Web search

We propose a new approach for indexing Web contents. The essence of our approach is that we use a psycho-linguistic word database for calculating the index of Web pages. Since there is no existing database designed for this purpose, we started our study by creating a word database. We will show that the database can be effectively used for estimating the reading levels of specific Web pages. We will also show that this approach can be used to reflect user profiles in Web searches.

Supervised semi-autoencoder learning for multi-layered neural networks

The present paper proposes a new type of layer-wise learning for multi-layered neural networks. Multi-layered neural networks have the serious problem of vanishing information, where information contained in input patterns is gradually lost, and preventing neural networks from learning input patterns. In particular, the autoencoders used in the greedy layer-wise pre-training tend to lose the original information by going through multiple layers. For this problem, we propose a new approach called “supervised semi-autoencoder” to solve the problem of vanishing information. The new method is close to the ordinary autoencoder, but the outputs are the original inputs and the corresponding targets for amplifying information in input patterns. In addition, the computational procedures are simplified by using potential learning, in which the potentiality of neurons is determined before learning, and is given as initial weights. Thus, the complicated adjustment of parameters is not necessary. The method was applied to the well-known crab data set as well as the real data set of eye-tracking records. In both cases, with relatively small-sized data, neural networks could find internal representations close to those obtained by larger ones, and better generalization performance could be observed.

Frequent pattern mining of eye-tracking records partitioned into cognitive chunks

Assuming that scenes would be visually scanned by chunking information, we partitioned fixation sequences of web page viewers into chunks using isolate gaze point(s) as the delimiter. Fixations were coded in terms of the segments in a 5 × 5 mesh imposed on the screen. The identified chunks were mostly short, consisting of one or two fixations. These were analyzed with respect to the within- and between-chunk distances in the overall records and the patterns (i.e., subsequences) frequently shared among the records. Although the two types of distances were both dominated by zero- and one-block shifts, the primacy of the modal shifts was less prominent between chunks than within them. The lower primacy was compensated by the longer shifts. The patterns frequently extracted at three threshold levels were mostly simple, consisting of one or two chunks. The patterns revealed interesting properties as to segment differentiation and the directionality of the attentional shifts.

An automatic web site menu structure evaluation

The purpose of this paper is to propose a method for automatically evaluating Web site menu structures. The evaluation system requires content data and a menu structure with link names. This approach consists of three stages. First, the system classifies the content data into appropriate links. Second, the system identifies the usability problems for all content data. Third, the system calculates an index that indicates the averaged predicted mouse clicks for the menu structure. As applications, a link name selection problem and a link structure evaluation problem are discussed. This system was also applied to real data, such as Encartas and Wikipedias menus. The results confirmed the usefulness of the system.

Generating Explicit Self-Organizing Maps by Information Maximization

In this paper, we propose a new information theoretic method for self-organizing maps. In realizing competition, neither the winner-all-take algorithm nor lateral inhibition is used. Instead, the new method is based upon mutual information maximization between input patterns and competitive units. Thus, competition processes are flexibly controlled to produce explicit self-organizing maps. We applied our method to a road classification problem. Experimental results confirmed that the new method could produce more explicit self-organizing maps than conventional self-organizing methods.