Kazumi Saito

Second-order Learning Algorithm with Squared Penalty Term

This article compares three penalty terms with respect to the efficiency of supervised learning, by using first- and second-order off-line learning algorithms and a first-order on-line algorithm. Our experiments showed that for a reasonably adequate penalty factor, the combination of the squared penalty term and the second-order learning algorithm drastically improves the convergence performance in comparison to the other combinations, at the same time bringing about excellent generalization performance. Moreover, in order to understand how differently each penalty term works, a function surface evaluation is described. Finally, we show how cross validation can be applied to find an optimal penalty factor.

Modeling user behavior in recommender systems based on maximum entropy

We propose a model for user purchase behavior in online stores that provide recommendation services. We model the purchase probability given recommendations for each user based on the maximum entropy principle using features that deal with recommendations and user interests. The proposed model enable us to measure the effect of recommendations on user purchase behavior, and the effect can be used to evaluate recommender systems. We show the validity of our model using the log data of an online cartoon distribution service, and measure the recommendation effects for evaluating the recommender system.

Recommendation Method for Improving Customer Lifetime Value

It is important for online stores to improve customer lifetime value (LTV) if they are to increase their profits. Conventional recommendation methods suggest items that best coincide with users interests to maximize the purchase probability, and this does not necessarily help improve LTV. We present a novel recommendation method that maximizes the probability of the LTV being improved, which can apply to both measured and subscription services. Our method finds frequent purchase patterns among high-LTV users and recommends items for a new user that simulate the found patterns. Using survival analysis techniques, we efficiently find the patterns from log data. Furthermore, we infer a users interests from the purchase history based on maximum entropy models and use the interests to improve recommendation. Since a higher LTV is the result of greater user satisfaction, our method benefits users as well as online stores. We evaluate our method using two sets of real log data for measured and subscription services.

Computational Discovery of Communicable Scientific Knowledge

In this paper we distinguish between two computational paradigms for knowledge discovery that share the notion of heuristic search, but differ in the importance they place on using scientific formalisms to state discovered knowledge. We also report progress on computational methods for discovering such communicable knowledge in two domains, one involving the regulation of photosynthesis in phytoplankton and the other involving carbon production by vegetation in the Earth ecosystem. In each case, we describe a representation for models, methods for using data to revise existing models, and some initial results. In closing, we discuss related work on the computational discovery of communicable scientific knowledge and outline directions for future research.

A constructive learning algorithm for an HME

A hierarchical mixtures of experts (HME) model has been applied to several classes of problems, and its usefulness has been shown. However, defining an adequate structure in advance is required and the resulting performance depends on the structure. To overcome this problem, a constructive learning algorithm for an HME is proposed; it includes an initialization method, a training method and an extension method. In our experiments, which used parity problems and a function approximation problem, the proposed algorithm worked much better than the conventional method.

Multinomial PCA for extracting major latent topics from document streams

We propose a new unsupervised learning method called multinomial PCA (MuPCA) for efficiently extracting the major latent topics from a document stream based on the bag-of-words (BOW) representation of a document. Unlike PCA, MuPCA follows a suitable probabilistic generative model for the document stream represented as time-series of word-frequency vectors. Using real data of document streams on the Web, we experimentally demonstrate the effectiveness of the proposed method.

Modeling of growing networks with communities

We propose a growing network model and its learning algorithm. Unlike the conventional scale-free models, we incorporate community structure, which is an important characteristic of many real-world networks including the Web. In our experiments, we confirmed that the proposed model exhibits a degree distribution with a power-law tail, and our method can precisely estimate the probability of a new link creation from data without community information. Moreover, by introducing a measure of dynamic hub-degrees, we could predict the change of hub-degrees between communities.

Parrot-like speaking using optimal vector quantization

Parrot-like speaking can be considered as one of the most fundamental abilities of humans or robots. It is not a transformation of a target speech signal, but a perception-and-action process: recognizing the target speech and producing a mimic one using a voice obtained from a voice owner. This paper presents a connectionist parrot-like speaking system. Our approach employs the record-and-edit approach with an acoustic wave segment as the processing unit, and uses a vector quantizer for two purposes: to build a segment database as a natural voice of a robot, and to cluster the segment database to speed up the mimicking. The experimental parrot system works mostly well, mimicking any target speech and sounding like a voice owner.

Weight sharing on naive Bayes document model

In this paper, we study weight sharing on the naive Bayes document model. Firstly we consider splitting words into a relatively small number of groups such that words in each group have the same parameter value. This problem can be regarded as a probabilistic parameter sharing task. In this task, we formalize the problem in terms of maximum likelihood estimation, and then propose an algorithm for this purpose. Secondly we focus on an adaptive hyperparameter estimation problem based on prior distributions constructed by using such word groups. This problem can be regarded as a hyperparameter sharing task. In this task, we describe a framework and algorithm, which enables to derive the unique optimal solution in the context of leave-one-out cross validation. In our experiments using a benchmark document set called webkb, we show a series of simulation results using the proposed algorithms.

MDL regularizer: a new regularizer based on the MDL principle

This paper proposes a new regularization method based on the MDL (minimum description length) principle. An adequate precision weight vector is trained by approximately truncating the maximum likelihood weight vector. The main advantage of the proposed regularizer over existing ones is that it automatically determines a regularization factor without assuming any specific prior distribution with respect to the weight values. Our experiments using a regression problem showed that the MDL regularizer significantly improves the generalization error of a second-order learning algorithm and shows a comparable generalization performance to the best tuned weight-decay regularizer.