Ibrahim Delibalta

Sequential Nonlinear Learning for Distributed Multiagent Systems via Extreme Learning Machines

We study online nonlinear learning over distributed multiagent systems, where each agent employs a single hidden layer feedforward neural network (SLFN) structure to sequentially minimize arbitrary loss functions. In particular, each agent trains its own SLFN using only the data that is revealed to itself. On the other hand, the aim of the multiagent system is to train the SLFN at each agent as well as the optimal centralized batch SLFN that has access to all the data, by exchanging information between neighboring agents. We address this problem by introducing a distributed subgradient-based extreme learning machine algorithm. The proposed algorithm provides guaranteed upper bounds on the performance of the SLFN at each agent and shows that each of these individual SLFNs asymptotically achieves the performance of the optimal centralized batch SLFN. Our performance guarantees explicitly distinguish the effects of data- and network-dependent parameters on the convergence rate of the proposed algorithm. The experimental results illustrate that the proposed algorithm achieves the oracle performance significantly faster than the state-of-the-art methods in the machine learning and signal processing literature. Hence, the proposed method is highly appealing for the applications involving big data.

Optimal and Efficient Distributed Online Learning for Big Data

We propose optimal and efficient distributed online learning strategies for Big Data applications. Here, we consider the optimal state estimation over distributed network of autonomous data sources. The autonomous data sources can generate and process data locally irrespective of any centralized control unit. We seek to enhance the learning rate through the distributed control of those autonomous data sources. We emphasize that although this problem attracted significant attention and extensively studied in different fields including services computing and machine learning disciplines, all the well-known strategies achieve sub optimal online learning performance in the mean square error sense. To this end, we introduce the oracle algorithm as the optimal distributed online learning strategy. We also propose the optimal and efficient distributed online learning algorithm that reduces the communication load tremendously, i.e., Requires the undirected disclosure of only a single scalar. Finally, we demonstrate the significant performance gains due to the proposed strategies with respect to the state-of-the-art approaches.

Efficient NP Tests for Anomaly Detection Over Birth-Death Type DTMCs

We propose computationally highly efficient Neyman-Pearson (NP) tests for anomaly detection over birth-death type discrete time Markov chains. Instead of relying on extensive Monte Carlo simulations (as in the case of the baseline NP), we directly approximate the log-likelihood density to match the desired false alarm rate; and therefore obtain our efficient implementations. The proposed algorithms are appropriate for processing large scale data in online applications with real time false alarm rate controllability. Since we do not require parameter tuning, our algorithms are also adaptive to non-stationarity in the data source. In our experiments, the proposed tests demonstrate superior detection power compared to the baseline NP while nearly achieving the desired rates with negligible computational resources.

Big data signal processing using boosted RLS algorithm

We propose an efficient method for the high dimensional data regression. To this end, we use a least mean squares (LMS) filter followed by a recursive least squares (RLS) filter and combine them via boosting notion extensively used in machine learning literature. Moreover, we provide a novel approach where the RLS filter is updated randomly in order to reduce the computational complexity while not giving up more on the performance. In the proposed algorithm, after the LMS filter produces an estimate, depending on the error made on this step, the algorithm decides whether or not updating the RLS filter. Since we avoid updating the RLS filter for all data sequence, the computational complexity is significantly reduced. Error performance and the computation time of our algorithm is demonstrated for a highly realistic scenario.

Online text classification for real life tweet analysis

In this paper, we study multi-class classification of tweets, where we introduce highly efficient dimensionality reduction techniques suitable for online processing of high dimensional feature vectors generated from freely-worded text. As for the real life case study, we work on tweets in the Turkish language, however, our methods are generic and can be used for other languages as clearly explained in the paper. Since we work on a real life application and the tweets are freely worded, we introduce text correction, normalization and root finding algorithms. Although text processing and classification are highly important due to many applications such as emotion recognition, advertisement selection, etc., online classification and regression algorithms over text are limited due to need for high dimensional vectors to represent natural text inputs. We overcome such limitations by showing that randomized projections and piecewise linear models can be efficiently leveraged to significantly reduce the computational cost for feature vector extraction from the tweets. Hence, we can perform multi-class tweet classification and regression in real time. We demonstrate our results over tweets collected from a real life case study where the tweets are freely-worded, e.g., with emoticons, shortened words, special characters, etc., and are unstructured. We implement several well-known machine learning algorithms as well as novel regression methods and demonstrate that we can significantly reduce the computational complexity with insignificant change in the classification and regression performance.

Online churn detection on high dimensional cellular data using adaptive hierarchical trees

We study online sequential logistic regression for churn detection in cellular networks when the feature vectors lie in a high dimensional space on a time varying manifold. We escape the curse of dimensionality by tracking the subspace of the underlying manifold using a hierarchical tree structure. We use the projections of the original high dimensional feature space onto the underlying manifold as the modified feature vectors. By using the proposed algorithm, we provide significant classification performance with significantly reduced computational complexity as well as memory requirement. We reduce the computational complexity to the order of the depth of the tree and the memory requirement to only linear in the intrinsic dimension of the manifold. We provide several results with real life cellular network data for churn detection.

Adaptive hierarchical space partitioning for online classification

We propose an online algorithm for supervised learning with strong performance guarantees under the empirical zero-one loss. The proposed method adaptively partitions the feature space in a hierarchical manner and generates a powerful finite combination of basic models. This provides algorithm to obtain a strong classification method which enables it to create a linear piecewise classifier model that can work well under highly non-linear complex data. The introduced algorithm also have scalable computational complexity that scales linearly with dimension of the feature space, depth of the partitioning and number of processed data. Through experiments we show that the introduced algorithm outperforms the state-of-the-art ensemble techniques over various well-known machine learning data sets.

Boosted LMS-based piecewise linear adaptive filters

We introduce the boosting notion extensively used in different machine learning applications to adaptive signal processing literature and implement several different adaptive filtering algorithms. In this framework, we have several adaptive constituent filters that run in parallel. For each newly received input vector and observation pair, each filter adapts itself based on the performance of the other adaptive filters in the mixture on this current data pair. These relative updates provide the boosting effect such that the filters in the mixture learn a different attribute of the data providing diversity. The outputs of these constituent filters are then combined using adaptive mixture approaches. We provide the computational complexity bounds for the boosted adaptive filters. The introduced methods demonstrate improvement in the performances of conventional adaptive filtering algorithms due to the boosting effect.

Online anomaly detection with constant false alarm rate

We propose a computationally highly scalable online anomaly detection algorithm for time series, which achieves with no parameter tuning a specified false alarm rate while minimizing the miss rate. The proposed algorithm sequentially operates on a fast streaming temporal data, extracts the nominal attributes under possibly varying Markov statistics and then declares an anomaly when the observations are statistically sufficiently deviant. Regardless of whether the source is stationary or non-stationary, our algorithm is guaranteed to closely achieve the desired false alarm rates at negligible computational costs. In this regard, the proposed algorithm is highly novel and appropriate especially for big data applications. Through the presented simulations, we demonstrate that our algorithm outperforms its competitor, i.e., the Neyman-Pearson test that relies on the Monte Carlo trials, even in the case of strong non-stationarity.

Computationally highly efficient mixture of adaptive filters

We introduce a new combination approach for the mixture of adaptive filters based on the set-membership filtering (SMF) framework. We perform SMF to combine the outputs of several parallel running adaptive algorithms and propose unconstrained, affinely constrained and convexly constrained combination weight configurations. Here, we achieve better trade-off in terms of the transient and steady-state convergence performance while providing significant computational reduction. Hence, through the introduced approaches, we can greatly enhance the convergence performance of the constituent filters with a slight increase in the computational load. In this sense, our approaches are suitable for big data applications where the data should be processed in streams with highly efficient algorithms. In the numerical examples, we demonstrate the superior performance of the proposed approaches over the state of the art using the well-known datasets in the machine learning literature.