Symeon Chouvardas

Adaptive Robust Distributed Learning in Diffusion Sensor Networks

In this paper, the problem of adaptive distributed learning in diffusion networks is considered. The algorithms are developed within the convex set theoretic framework. More specifically, they are based on computationally simple geometric projections onto closed convex sets. The paper suggests a novel combine-project-adapt protocol for cooperation among the nodes of the network; such a protocol fits naturally with the philosophy that underlies the projection-based rationale. Moreover, the possibility that some of the nodes may fail is also considered and it is addressed by employing robust statistics loss functions. Such loss functions can easily be accommodated in the adopted algorithmic framework; all that is required from a loss function is convexity. Under some mild assumptions, the proposed algorithms enjoy monotonicity, asymptotic optimality, asymptotic consensus, strong convergence and linear complexity with respect to the number of unknown parameters. Finally, experiments in the context of the system-identification task verify the validity of the proposed algorithmic schemes, which are compared to other recent algorithms that have been developed for adaptive distributed learning.

A Sparsity Promoting Adaptive Algorithm for Distributed Learning

In this paper, a sparsity promoting adaptive algorithm for distributed learning in diffusion networks is developed. The algorithm follows the set-theoretic estimation rationale. At each time instance and at each node of the network, a closed convex set, known as property set, is constructed based on the received measurements; this defines the region in which the solution is searched for. In this paper, the property sets take the form of hyperslabs. The goal is to find a point that belongs to the intersection of these hyperslabs. To this end, sparsity encouraging variable metric projections onto the hyperslabs have been adopted. In addition, sparsity is also imposed by employing variable metric projections onto weighted l1 balls. A combine adapt cooperation strategy is adopted. Under some mild assumptions, the scheme enjoys monotonicity, asymptotic optimality and strong convergence to a point that lies in the consensus subspace. Finally, numerical examples verify the validity of the proposed scheme compared to other algorithms, which have been developed in the context of sparse adaptive learning.

Placing dynamic content in caches with small population

This paper addresses a fundamental limitation for the adoption of caching for wireless access networks due to small population sizes. This shortcoming is due to two main challenges: making timely estimates of varying content popularity and inferring popular content from small samples. We propose a framework which alleviates such limitations. To timely estimate varying popularity in a context of a single cache we propose an Age-Based Threshold (ABT) policy which caches all contents requested more times than a threshold N (τ), where τ is the content age. We show that ABT is asymptotically hit rate optimal in the many contents regime, which allows us to obtain the first characterization of the optimal performance of a caching system in a dynamic context. We then address small sample sizes focusing on L local caches and one global cache. On the one hand we show that the global cache learns L times faster by aggregating all requests from local caches, which improves hit rates. On the other hand, aggregation washes out local characteristics of correlated traffic which penalizes hit rate. This motivates coordination mechanisms which combine global learning of popularity scores in clusters and Least-Recently-Used (LRU) policy with prefetching.

Trading off Complexity With Communication Costs in Distributed Adaptive Learning via Krylov Subspaces for Dimensionality Reduction

In this paper, the problem of dimensionality reduction in adaptive distributed learning is studied. We consider a network obeying the ad-hoc topology, in which the nodes sense an amount of data and cooperate with each other, by exchanging information, in order to estimate an unknown, common, parameter vector. The algorithm, to be presented here, follows the set-theoretic estimation rationale; i.e., at each time instant and at each node of the network, a closed convex set is constructed based on the received measurements, and this defines the region in which the solution is searched for. In this paper, these closed convex sets, known as property sets, take the form of hyperslabs. Moreover, in order to reduce the number of transmitted coefficients, which is dictated by the dimension of the unknown vector, we seek for possible solutions in a subspace of lower dimension; the technique will be developed around the Krylov subspace rationale. Our goal is to find a point that belongs to the intersection of this infinite number of hyperslabs and the respective Krylov subspaces. This is achieved via a sequence of projections onto the property sets and the Krylov subspaces. The case of highly correlated inputs that degrades the performance of the algorithm is also considered. This is overcome via a transformation which whitens the input. The proposed schemes are brought in a decentralized form by adopting the combine-adapt cooperation strategy among the nodes. Full convergence analysis is carried out and numerical tests verify the validity of the proposed schemes in different scenarios in the context of the adaptive distributed system identification task.

Distributed robust labeling of audio sources in heterogeneous wireless sensor networks

A novel algorithm for distributed labeling of speech sources is proposed. We consider a wireless sensor network comprising devices that are equipped with multiple microphones, which can “hear” a number of speech signals. The labeling task is performed in a decentralized fashion with a new two-step approach. The first step corresponds to the distributed extraction of proper source-specific features from the mixed signals. In the second step, these features are exploited via a distributed unsupervised learning technique. We present approaches that can be used in hierarchically organized or in non-hierarchically organized network configurations. Numerical examples using real data display the performance of the proposed technique.

Robust Subspace Tracking With Missing Entries: The Set-Theoretic Approach

In this paper, an Adaptive Projected Subgradient Method (APSM) based algorithm for robust subspace tracking is introduced. A properly chosen cost function is constructed at each time instance and the goal is to seek for points, which belong to the zero level set of this function; i.e., the set of points which score a zero loss. At each iteration, an outlier detection mechanism is employed, in order to conclude whether the current data vector contains outlier noise or not. In the sequel, a sparsity-promoting greedy algorithm is employed for the outlier vector estimation allowing the purification of the corrupted data from the outlier noise, prior to any further processing. Furthermore, the case where the observation vectors are partially observed is attacked via a prediction procedure, which estimates the values of the unobserved (missing) coefficients. A theoretical analysis is carried out and the simulation experiments, within the contexts of robust subspace estimation and robust matrix completion, demonstrate the enhanced performance of the proposed scheme compared to recently developed state of the art algorithms.

A diffusion kernel LMS algorithm for nonlinear adaptive networks

This work presents a distributed algorithm for nonlinear adaptive learning. In particular, a set of nodes obtain measurements, sequentially one per time step, which are related via a nonlinear function; their goal is to collectively minimize a cost function by employing a diffusion based Kernel Least Mean Squares (KLMS). The algorithm follows the Adapt Then Combine mode of cooperation. Moreover, the theoretical properties of the algorithm are studied and it is proved that under certain assumptions the algorithm suffers a no regret bound. Finally, comparative experiments verify that the proposed scheme outperforms other variants of the LMS.

Stochastic Analysis of Hyperslab-Based Adaptive Projected Subgradient Method Under Bounded Noise

This letter establishes a novel analysis of the Adaptive Projected Subgradient Method (APSM) in the intersection of the stochastic and robust estimation paradigms. Utilizing classical worst-case bounds on the noise process, drawn from the robust estimation methodology, the present study demonstrates that the hyperslab-inspired version of the APSM generates a sequence of estimates which converges to a point located, with probability one, arbitrarily close to the estimand. Numerical tests and comparisons with classical time-adaptive algorithms corroborate the theoretical findings of the study.

An online algorithm for distributed dictionary learning

This paper proposes a novel algorithm for online distributed dictionary learning, where a set of nodes is requested to collectively estimate a common dictionary via sequentially received data vectors. At each time instance, in which a new datum becomes available, the sparse representation of the data with respect to the estimated dictionary is computed locally at each node by employing a sparsity promoting algorithm. In the sequel, the nodes cooperate in order to collaboratively update the dictionary via the distributed Recursive Least Squares (RLS) algorithm. Numerical examples, both with synthetic and real data, validate that the performance of the proposed algorithm is comparable to that of centralized state of the art algorithms.

An Adaptive Projected Subgradient based algorithm for robust subspace tracking

In this paper, an Adaptive Projected Subgradient Method (APSM) based algorithm for robust subspace tracking is introduced. A properly chosen cost function is constructed at each time instance and the goal is to seek for points, which belong to the zero level set of this function; i.e., the set of points which score a zero loss. In each iteration, an outlier detection mechanism is employed, in order to conclude whether the current data vector contains outlier noise or not. Furthermore, a sparsity-promoting greedy algorithm is employed for the outlier vector estimation allowing the purification of the corrupted data from the outlier noise prior further processing. A theoretical analysis is carried out and experiments within the context of robust subspace estimation exhibit the enhanced performance of the proposed scheme compared to a recently developed state of the art algorithm.