Goktug T. Cinar

Hidden state estimation using the Correntropy Filter with fixed point update and adaptive kernel size

In this paper we review the Correntropy Filter for hidden state estimation and we introduce the fixed point update rule for the Correntropy Filter instead of using gradient ascent for faster convergence. We further propose an adaptive kernel bandwidth selection algorithm. It is shown that the new filter outperforms the Kalman Filter and has no free parameters. The algorithms capabilities are demonstrated on a simulated experiment and a vehicle tracking problem.

Adaptive background estimation using an information theoretic cost for hidden state estimation

Hidden state estimation in linear systems is a popular and broad research topic which became a mainstream research area after Rudolf Kalmans seminal paper. The Kalman Filter (KF) gives the optimal solution to the estimation problem in a setting where all the processes are Gaussian random processes. However because of the suboptimal behavior of the KF in non-Gaussian settings, there is a need for a new filter that can extract higher order information from the signals. In this paper we propose using an information theoretic cost function utilizing the similarity measure Correntropy as a performance index. This results in a different perspective on hidden state estimation. We present the superior performance of the new filter on both synthetic data and on adaptive background estimation problem and discuss future research directions.

Clustering of time series using a hierarchical linear dynamical system

The auditory cortex in the brain does effortlessly a better job of extracting information from the acoustic world than our current generation of signal processing algorithms. Abstracting the principles of the auditory cortex, the proposed architecture is based on Kalman filters with hierarchically coupled state models that stabilize the input dynamics and provide a representation space. This approach extracts information from the input and self-organizes it in the higher layers leading to an algorithm capable of clustering time series in an unsupervised manner. An important characteristic of the methodology is that it is adaptive and self-organizing, i.e. previous exposure to the acoustic input is the only requirement for learning and recognition, so there is no need of selecting the number of clusters.

Effective insect recognition using a stacked autoencoder with maximum correntropy criterion

Throughout the history, insects had been intimately connected to humanity, in both positive and negative ways. Insects play an important part in crop pollination, on the other hand, some of them spread diseases that kill millions of people every year. Effective control of harmful insects while having little impact to beneficial insects and environment is extremely important. Recently, an intelligent trap that uses laser sensors was presented to control the population of target insects. The device could record and analyze sensor signals when an insect passes through the trap and make quick decisions whether to catch it or not. The effectiveness of the trap relies on the correct choice of classification algorithm to perform the insect detection. In this paper, we propose to use a deep neural network with maximum correntropy criterion (MCC) for reliable classification of insects in real-time. Experimental results show that, deep networks are effective for learning stable features from brief insect passage signals. By replacing the mean square error cost with MCC, the robustness of autoencoders against noise is improved significantly and robust features could be learned. The method is tested on five species of insects and a total of 5325 passages. High classification accuracy of 92.1% is achieved. Compared with previously applied methods, better classification performance is obtained using only 10% of the computation time. Therefore, our method is efficient and reliable for online insect detection.

Hierarchical Linear Dynamical Systems: A new model for clustering of time series

The auditory cortex in the brain does effortlessly a better job of extracting information from the acoustic world than our current generation of signal processing algorithms. The proposed architecture, Hierarchical Linear Dynamical System (HLDS), is based on Kalman filters with hierarchically coupled state models that stabilize the input dynamics and provide a representation space. This approach extracts information from the input and self-organizes it in the higher layers leading to an algorithm capable of clustering time series in an unsupervised manner. In this paper we further investigate the properties of HLDS, demonstrate its performance on music rather than isolated notes and propose the time domain implementation to overcome one of its current bottlenecks.

A study of musical pitch distance using a self-organized hierarchical linear dynamical system on acoustic signals

The hierarchical linear dynamical system (HLDS) is a self-organizing architecture to cluster acoustic time series. The HLDS architecture is equivalent to a Kalman filter whose top-layer state learns to create subspaces that tessellate the acoustic signal in regions that correspond to different musical pitches. The observation layer of the HLDS is built from a biologically plausible gammatone filter bank that provides the representation space for the state assignments. An important characteristic of the methodology is that it is adaptive and self-organizing, i.e., previous exposure to the acoustic input is the only requirement for learning and recognition. In this article we show that the representation space that the algorithm learns from acoustic signals preserves the organization found in monophonic notes, and exhibits (for isolated pitches and triads) properties suggested in the theory of efficient chromatic voice leading and neo-Riemannian theories.

Hierarchical linear dynamical systems for unsupervised musical note recognition

Abstract In this paper we develop a new framework for time series segmentation based on a Hierarchical Linear Dynamical System (HLDS), and test its performance on monophonic and polyphonic musical note recognition. The center piece of our approach is the inclusion of constraints in the filter topology, instead of on the cost function as normally done in machine learning. Just by slowing down the dynamics of the top layer of an augmented (multilayer) state model, which is still compatible with the recursive update equation proposed originally by Kalman, the system learns directly from data all the musical notes, without labels, effectively creating a time series clustering algorithm that does not require segmentation. We analyze the HLDS properties and show that it provides better classification accuracy compared to current state-of-the-art approaches.

Parallel flow in Deep Predictive Coding Networks

This paper proposes a cognitive architecture for sensory processing of multimodal data. The cognitive architectures, referred to as Deep Predictive Coding Networks (DPCN) were first used to model video streams. Here we use DPCNs with two input sources, for example: video and speech recordings. We train DPCNs as generative models of both sensors. Since we constrain the network to have a single hidden code for both inputs, we name the proposed architecture as Multimodal DPCN (MDPCN). Experimental results show that the “parallel” flow between the two sensory modes increases the interclass separability achieved by unsupervised clustering. We validate the proposed method with a multimodal classification task using part of the VIDTIMIT dataset.

Pitch estimation using non-negative matrix factorization

The problem of pitch detection consists of estimating the dominant frequency present in a certain time window. This paper demonstrates and analyzes the use of a non-negative matrix factorization technique with a frequency basis formed with a correntropy kernel. This offers the advantage that the frequency basis is adaptable, allowing the matrix factorization to fit the data precisely, as well as including a dictionary specifically to account for noise. Using non-negative matrix factorization also allows an increase in dimensionality, which increases the frequency resolution of the algorithm. The method is tested on a database of trumpet notes and compared to other current methods, improving on their performance for noisy signals.

Sequential causal estimation and learning from time-varying images

Dynamic models are used in modeling the perceptual systems with hierarchies. But most of the models assume Gaussian statistics on the underlying causes. In this paper we try to develop a basic building block for these hierarchical models where the causes are assumed to be non-Gaussian. We describe a sequential dual estimation framework for inferring the hidden states and unknown causes/inputs while learning the parameters of the model. It is observed that the algorithm is able to extract bases from the time varying image sequence that resembles receptive fields of the simple cells in V1. In addition, the dynamical model gives us the ability to deconvolve spatial and temporal changes in the image sequence.