Pavel Polyakov

Comparison of neural network Boolean factor analysis method with some other dimension reduction methods on bars problem

In this paper, we compare performance of novel neural network based algorithmfor Boolean factor analysiswith several dimension reduction techniques as a tool for feature extraction. Compared are namely singular value decomposition, semi-discrete decomposition and non-negative matrix factorization algorithms, including some cluster analysis methods as well. Even if the mainly mentioned methods are linear, it is interesting to compare them with neural network based Boolean factor analysis, because they arewell elaborated. Second reason for this is to show basic differences between Boolean and linear case. So called bars problem is used as the benchmark. Set of artificial signals generated as a Boolean sum of given number of bars is analyzed by thesemethods. Resulting images show that Boolean factor analysis is upmost suitable method for this kind of data.

Bars problem solving - new neural network method and comparison

Bars problem is widely used as a benchmark for the class of feature extraction tasks. In this model, artificial data set is generated as a Boolean sum of a given number of bars. We show that the most suitable technique for feature set extraction in this case is neural network based Boolean factor analysis. Results are confronted with several dimension reduction techniques. These are singular value decomposition, semi-discrete decomposition and non-negative matrix factorization. Even if these methods are linear, it is interesting to compare them with neural network attempt, because they are well elaborated and are often used for a similar tasks. We show that frequently used cluster analysis methods can bring interesting results, at least for first insight to the data structure.

New Neural Network Based Approach Helps to Discover Hidden Russian Parliament Voting Patterns

The sparse encoded Hopfield like neural network is modified to provide the Boolean factor analysis. New, more efficient method of sequential factor extraction, based on the characteristics behavior of the Lyapunov function is introduced. Efficiency of this attempt is shown not only on simulated data but on real data from Russian parliament but as well.

Two Expectation-Maximization algorithms for Boolean Factor Analysis

Methods for the discovery of hidden structures of high-dimensional binary data are one of the most important challenges facing the community of machine learning researchers. There are many approaches in the literature that try to solve this hitherto rather ill-defined task. In the present study, we propose a general generative model of binary data for Boolean Factor Analysis and introduce two new Expectation-Maximization Boolean Factor Analysis algorithms which maximize the likelihood of a Boolean Factor Analysis solution. To show the maturity of our solutions we propose an informational measure of Boolean Factor Analysis efficiency. Using the so-called bars problem benchmark, we compare the efficiencies of the proposed algorithms to that of Dendritic Inhibition Neural Network, Maximal Causes Analysis, and Boolean Matrix Factorization. Last mentioned methods were taken as related methods as they are supposed to be the most efficient in bars problem benchmark. Then we discuss the peculiarities of the two methods we proposed and the three related methods in performing Boolean Factor Analysis.

Estimation of Boolean Factor Analysis Performance by Informational Gain

To evaluate the soundness of multidimensional binary signal analysis based on Boolean factor analysis theory and mainly of its neural network implementation, proposed is a universal measure - informational gain. This measure is derived using classical informational theory results. Neural network based Boolean factor analysis method efficiency is demonstrated using this measure, both when applied to Bars Problem benchmark data and to real textual data. It is shown that when applied to the well defined Bars Problem data, Boolean factor analysis provides informational gain close to its maximum, i.e. the latent structure of the testing images data was revealed with the maximal accuracy. For scientific origin real textual data the informational gain provided by the method happened to be much higher comparing to that based on human experts proposal.

Clustering variables by classical approaches and neural network Boolean factor analysis

In this paper, we compare three methods for grouping of binary variables: neural network Boolean factor analysis, hierarchical clustering, and a linear factor analysis on the mushroom dataset. In contrast to the latter two traditional methods, the advantage of neural network Boolean factor analysis is its ability to reveal overlapping classes in the dataset. It is shown that the mushroom dataset provides a good demonstration of this advantage because it contains both disjunctive and overlapping classes.

Image Analysis by Methods of Dimension Reduction

In this paper, we compare performance of several dimension reduction techniques, namely LSI, NMF, SDD, Boolean factor analysis, and cluster analysis. The qualitative comparison is evaluated on a collection of bars. We compare the quality of these methods from on the base of the visual impact.

Origin and elimination of two global spurious attractors in Hopfield-like neural network performing Boolean factor analysis

Factor analysis is used in a number of applications. One example is image recognition, where it is often necessary to learn representations of the underlying components of images, such as objects, object-parts, or features. Another example is data compression when original data is transformed into a space of lower dimension. The goal of factor analysis is to find the underlying factors (factor loadings) and the contributions of these factors into the original observations (factor scores). Recently, we have proposed the method of Boolean factor analysis based on the ability of the Hopfield-like network to create attractors for factors [19]. It shows that an obstacle to using this network for Boolean factor analysis is the appearance of two global spurious attractors that have no relation to internal structure of analyzed signals. To eliminate these attractors we had to modify the common architecture of Hopfield network, adding a special inhibitory neuron. The existence of two global attractors and their elimination by the special inhibitory neuron were illustrated by Frolov et al. [19] only by some computer simulations. Since the appearance of those attractors is a novel important phenomenon, in this paper we investigate it both analytically and by additional computer simulations, to prove its validity, and explain its origin.

New BFA method based on attractor neural network and likelihood maximization

What is suggested is a new approach to Boolean factor analysis, which is an extension of the previously proposed Boolean factor analysis method: Hopfield-like attractor neural network with increasing activity. We increased its applicability and robustness when complementing this method by a maximization of the learning set likelihood function defined according to the Noisy-OR generative model. We demonstrated the efficiency of the new method using the data set generated according to the model. Successful application of the method to the real data is shown when analyzing the data from the Kyoto Encyclopedia of Genes and Genomes database which contains full genome sequencing for 1368 organisms.

Expectation-maximization approach to Boolean factor analysis

Methods for hidden structure of high-dimensional binary data discovery are one of the most important challenges facing machine learning community researchers. There are many approaches in literature that try to solve this hitherto rather ill-defined task. In the present study, we propose a most general generative model of binary data for Boolean factor analysis and introduce new Expectation-Maximization Boolean Factor Analysis algorithm which maximizes likelihood of Boolean Factor Analysis solution. Using the so-called bars problem benchmark, we compare efficiencies of Expectation-Maximization Boolean Factor Analysis algorithm with Dendritic Inhibition neural network. Then we discuss advantages and disadvantages of both approaches as regards results quality and methods efficiency.