Sergios Theodoridis

A Novel Efficient Cluster-Based MLSE Equalizer for Satellite Communication Channels with

In satellites, nonlinear amplifiers used near saturation severely distort the transmitted signal and cause difficulties in its reception. Nevertheless, the nonlinearities introduced by memoryless bandpass amplifiers preserve the symmetries of the-ary quadrature amplitude modulation (-QAM) constellation. In this paper, a cluster-based sequence equalizer (CBSE) that takes advantage of these symmetries is presented. The proposed equalizer exhibits enhanced performance compared to other techniques, including the conventional linear transversal equalizer, Volterra equalizers, and RBF network equalizers. Moreover, this gain in performance is obtained at a substantially lower computational cost.

Efficient least squares adaptive algorithms for FIR transversal filtering

A unified view of algorithms for adaptive transversal FIR filtering and system identification has been presented. Wiener filtering and stochastic approximation are the origins from which all the algorithms have been derived, via a suitable choice of iterative optimization schemes and appropriate design parameters. Following this philosophy, the LMS algorithm and its offspring have been presented and interpreted as stochastic approximations of iterative deterministic steepest descent optimization schemes. On the other hand, the RLS and the quasi-RLS algorithms, like the quasi-Newton, the FNTN, and the affine projection algorithm, have been derived as stochastic approximations of iterative deterministic Newton and quasi-Newton methods. Fast implementations of these methods have been discussed. Block-adaptive, and block-exact adaptive filtering have also been considered. The performance of the adaptive algorithms has been demonstrated by computer simulations.

Adaptive Learning in a World of Projections

This article presents a general tool for convexly constrained parameter/function estimation both for classification and regression tasks, in a timeadaptive setting and in (infinite dimensional) reproducing kernel Hilbert spaces (RKHS). The thematical framework is that of the set theoretic estimation formulation and the classical projections onto convex sets (POCS) theory. However, in contrast to the classical POCS methodology, which assumes a finite number of convex sets, our method builds upon our recent extension of the theory, which considers an infinite number of convex sets. Such a context is necessary to cope with the adaptive setting rationale, where data arrive sequentially. This articles goal is to review the advances that have taken place in this area over the years and present them, in simple geometric arguments, as an integral part and natural evolution of the classical POCS methodology. The structure of the resulting algorithms is such that it allows extension to general RKHS. In this perspective, two very powerful techniques, convex optimization and (implicit) mapping to RKHS, are combined, which provide a framework for a unifying treatment of linear and nonlinear modeling of both classification and regression tasks. Typical signal processing problems, such as filtering, smoothing, equalization, and beamforming, fall under this common umbrella. The methodology allows for the incorporation of a set of convex constraints, which encode a priori information. Convexity, rather than differentiability, is the only prerequisite for adopting error measures that quantify the models fit against a set of training data points. Moreover, the complexity per iteration step remains linear with respect to the number of unknown parameters. The potential of the theory is demonstrated via numerical simulations for two typical problems; adaptive equalization and adaptive robust beamforming.

Online Sparse System Identification and Signal Reconstruction Using Projections Onto Weighted

This paper presents a novel projection-based adaptive algorithm for sparse signal and system identification. The sequentially observed data are used to generate an equivalent sequence of closed convex sets, namely hyperslabs. Each hyperslab is the geometric equivalent of a cost criterion, that quantifies “data mismatch.” Sparsity is imposed by the introduction of appropriately designed weighted ℓ1 balls and the related projection operator is also derived. The algorithm develops around projections onto the sequence of the generated hyperslabs as well as the weighted ℓ1 balls. The resulting scheme exhibits linear dependence, with respect to the unknown systems order, on the number of multiplications/additions and an O(Llog2L) dependence on sorting operations, where L is the length of the system/signal to be estimated. Numerical results are also given to validate the performance of the proposed method against the Least-Absolute Shrinkage and Selection Operator (LASSO) algorithm and two very recently developed adaptive sparse schemes that fuse arguments from the LMS/RLS adaptation mechanisms with those imposed by the lasso rational.

\ell_{1}

In this paper, the problem of adaptive distributed learning in diffusion networks is considered. The algorithms are developed within the convex set theoretic framework. More specifically, they are based on computationally simple geometric projections onto closed convex sets. The paper suggests a novel combine-project-adapt protocol for cooperation among the nodes of the network; such a protocol fits naturally with the philosophy that underlies the projection-based rationale. Moreover, the possibility that some of the nodes may fail is also considered and it is addressed by employing robust statistics loss functions. Such loss functions can easily be accommodated in the adopted algorithmic framework; all that is required from a loss function is convexity. Under some mild assumptions, the proposed algorithms enjoy monotonicity, asymptotic optimality, asymptotic consensus, strong convergence and linear complexity with respect to the number of unknown parameters. Finally, experiments in the context of the system-identification task verify the validity of the proposed algorithmic schemes, which are compared to other recent algorithms that have been developed for adaptive distributed learning.

Balls

In this paper, we propose a novel technique for word spotting in historical printed documents combining synthetic data and user feedback. Our aim is to search for keywords typed by the user in a large collection of digitized printed historical documents. The proposed method consists of the following stages: (1) creation of synthetic image words; (2) word segmentation using dynamic parameters; (3) efficient feature extraction for each word image and (4) a retrieval procedure that is optimized by user feedback. Experimental results prove the efficiency of the proposed approach.

Adaptive Robust Distributed Learning in Diffusion Sensor Networks

A class of adaptive algorithms for the estimation of FIR (finite impulse response) transversal filters is presented. The main characteristic of this class is the fast computation of the gain vector needed for the adaptation of the transversal filters. The method for deriving these algorithms is based on the assumption that the input signal is autoregressive of order M, where M can be much smaller than the order of the filter to be estimated. Under this assumption the covariance matrix of the input signal is estimated by extending in a min-max way the M order sample covariance matrix. This estimate can be regarded as a generalization of the diagonal covariance matrix used in LMS and leads to an efficient computation of the gain needed for the adaptation. The new class of algorithms contains the LMS and the fast versions of LS as special cases. The complexity changes linearly with M, starting from the complexity of the LMS (for M=0) and ending at the complexity of the fast versions of LS. >

Keyword-guided word spotting in historical printed documents using synthetic data and user feedback

The goal of this paper is to derive a novel online algorithm for classification in reproducing kernel hilbert spaces (RKHS) by exploiting projection-based adaptive filtering tools. The paper brings powerful convex analytic and set theoretic estimation arguments in machine learning by revisiting the standard kernel-based classification as the problem of finding a point which belongs to a closed halfspace (a special closed convex set) in an RKHS. In this way, classification in an online setting, where data arrive sequentially, is viewed as the problem of finding a point (classifier) in the nonempty intersection of an infinite sequence of closed halfspaces in the RKHS. Convex analysis is also used to introduce sparsification arguments in the design by imposing an additional simple convex constraint on the norm of the classifier. An algorithmic solution to the resulting optimization problem, where new convex constraints are added every time instant, is given by the recently introduced adaptive projected subgradient method (APSM), which generalizes a number of well-known projection-based adaptive filtering algorithms such as the classical normalized least mean squares (NLMS) and the affine projection algorithm (APA). Under mild conditions, the generated sequence of estimates enjoys monotone approximation, strong convergence, asymptotic optimality, and a characterization of the limit point. Further, we show that the additional convex constraint on the norm of the classifier naturally leads to an online sparsification of the resulting kernel series expansion. We validate the proposed design by considering the adaptive equalization problem of a nonlinear channel, and by comparing it with classical as well as with recently developed stochastic gradient descent techniques.

Fast Newton transversal filters-a new class of adaptive estimation algorithms

Over the last decade, kernel methods for nonlinear processing have successfully been used in the machine learning community. The primary mathematical tool employed in these methods is the notion of the reproducing kernel Hilbert space (RKHS). However, so far, the emphasis has been on batch techniques. It is only recently, that online techniques have been considered in the context of adaptive signal processing tasks. Moreover, these efforts have only been focussed on real valued data sequences. To the best of our knowledge, no adaptive kernel-based strategy has been developed, so far, for complex valued signals. Furthermore, although the real reproducing kernels are used in an increasing number of machine learning problems, complex kernels have not, yet, been used, in spite of their potential interest in applications that deal with complex signals, with Communications being a typical example. In this paper, we present a general framework to attack the problem of adaptive filtering of complex signals, using either real reproducing kernels, taking advantage of a technique called complexification of real RKHSs, or complex reproducing kernels, highlighting the use of the complex Gaussian kernel. In order to derive gradients of operators that need to be defined on the associated complex RKHSs, we employ the powerful tool of Wirtingers Calculus, which has recently attracted attention in the signal processing community. Wirtingers calculus simplifies computations and offers an elegant tool for treating complex signals. To this end, in this paper, the notion of Wirtingers calculus is extended, for the first time, to include complex RKHSs and use it to derive several realizations of the complex kernel least-mean-square (CKLMS) algorithm. Experiments verify that the CKLMS offers significant performance improvements over several linear and nonlinear algorithms, when dealing with nonlinearities.

Online Kernel-Based Classification Using Adaptive Projection Algorithms

Filter bank-based multicarrier communications (FBMC) have recently attracted increased interest in both wired (e.g., xDSL, PLC) and wireless (e.g., cognitive radio) applications, due to their enhanced flexibility, higher spectral efficiency, and better spectral containment compared to conventional OFDM. A particular type of FBMC, the so-called FBMC/OQAM or OFDM/OQAM system, consisting of pulse shaped OFDM carrying offset QAM (OQAM) symbols, has received increasing attention due to, among other features, its higher spectral efficiency and implementation simplicity. It suffers, however, from an imaginary inter-carrier/inter-symbol interference that complicates signal processing tasks such as channel estimation. This paper focuses on channel estimation for OFDM/OQAM systems based on a known preamble. A review of the existing preamble structures and associated channel estimation methods is given, for both single- (SISO) and multiple-antenna (MIMO) systems. The various preambles are compared via simulations in both mildly and highly frequency selective channels.