Masahiro Yukawa

A sparse adaptive filtering using time-varying soft-thresholding techniques

In this paper, we propose a novel adaptive filtering algorithm based on an iterative use of (i) the proximity operator and (ii) the parallel variable-metric projection. Our time-varying cost function is a weighted sum of squared distances (in a variable-metric sense) plus a possibly nonsmooth penalty term, and the proposed algorithm is derived along the idea of proximal forward-backward splitting in convex analysis. For application to sparse-system identification problems, we employ the (weighted) ℓ1 norm as the penalty term, leading to a time-varying soft-thresholding operator. As the simple example of the proposed algorithm, we present the variable-metric affine projection algorithm composed with the time-varying soft-thresholding operator. Numerical examples demonstrate that the proposed algorithms notably outperform their counterparts without soft-thresholding both in convergence speed and steady-state mismatch, while the extra computational complexity due to the additional soft-thresholding is negligibly low.

Multikernel Adaptive Filtering

This paper exemplifies that the use of multiple kernels leads to efficient adaptive filtering for nonlinear systems. Two types of multikernel adaptive filtering algorithms are proposed. One is a simple generalization of the kernel normalized least mean square (KNLMS) algorithm [2], adopting a coherence criterion for dictionary designing. The other is derived by applying the adaptive proximal forward-backward splitting method to a certain squared distance function plus a weighted block l1 norm penalty, encouraging the sparsity of an adaptive filter at the block level for efficiency. The proposed multikernel approach enjoys a higher degree of freedom than those approaches which design a kernel as a convex combination of multiple kernels. Numerical examples show that the proposed approach achieves significant gains particularly for nonstationary data as well as insensitivity to the choice of some design-parameters.

Minimizing the moreau envelope of nonsmooth convex functions over the fixed point set of certain quasi-nonexpansive mappings

The first aim of this paper is to present a useful toolbox of quasi-nonexpansive mappings for convex optimization from the viewpoint of using their fixed point sets as constraints. Many convex optimization problems have been solved through elegant translations into fixed point problems. The underlying principle is to operate a certain quasi-nonexpansive mapping T iteratively and generate a convergent sequence to its fixed point. However, such a mapping often has infinitely many fixed points, meaning that a selection from the fixed point set Fix(T) should be of great importance. Nevertheless, most fixed point methods can only return an “unspecified” point from the fixed point set, which requires many iterations. Therefore, based on common sense, it seems unrealistic to wish for an “optimal” one from the fixed point set. Fortunately, considering the collection of quasi-nonexpansive mappings as a toolbox, we can accomplish this challenging mission simply by the hybrid steepest descent method, provided that the cost function is smooth and its derivative is Lipschitz continuous. A question arises: how can we deal with “nonsmooth” cost functions? The second aim is to propose a nontrivial integration of the ideas of the hybrid steepest descent method and the Moreau–Yosida regularization, yielding a useful approach to the challenging problem of nonsmooth convex optimization over Fix(T). The key is the use of smoothing of the original nonsmooth cost function by its Moreau–Yosida regularization whose the derivative is always Lipschitz continuous. The field of application of hybrid steepest descent method can be extended to the minimization of the ideal smooth approximation Fix(T). We present the mathematical ideas of the proposed approach together with its application to a combinatorial optimization problem: the minimal antenna-subset selection problem under a highly nonlinear capacity-constraint for efficient multiple input multiple output (MIMO) communication systems.

Adaptive Reduced-Rank Constrained Constant Modulus Algorithms Based on Joint Iterative Optimization of Filters for Beamforming

This paper proposes a robust reduced-rank scheme for adaptive beamforming based on joint iterative optimization (JIO) of adaptive filters. The novel scheme is designed according to the constant modulus (CM) criterion subject to different constraints. The proposed scheme consists of a bank of full-rank adaptive filters that forms the transformation matrix, and an adaptive reduced-rank filter that operates at the output of the bank of filters to estimate the desired signal. We describe the proposed scheme for both the direct-form processor (DFP) and the generalized sidelobe canceller (GSC) structures. For each structure, we derive stochastic gradient (SG) and recursive least squares (RLS) algorithms for its adaptive implementation. The Gram-Schmidt (GS) technique is applied to the adaptive algorithms for reformulating the transformation matrix and improving the performance. An automatic rank selection technique is developed and employed to determine the most adequate rank for the derived algorithms. A detailed complexity study and a convexity analysis are carried out. Simulation results show that the proposed algorithms outperform the existing full-rank and reduced-rank methods in convergence and tracking performance.

Pairwise Optimal Weight Realization—Acceleration Technique for Set-Theoretic Adaptive Parallel Subgradient Projection Algorithm

The adaptive parallel subgradient projection (PSP) algorithm was proposed in 2002 as a set-theoretic adaptive filtering algorithm providing fast and stable convergence, robustness against noise, and low computational complexity by using weighted parallel projections onto multiple time-varying closed half-spaces. In this paper, we present a novel weighting technique named pairwise optimal weight realization (POWER) for further acceleration of the adaptive PSP algorithm. A simple closed-form formula is derived to compute the projection onto the intersection of two closed half-spaces defined by a triplet of vectors. Using the formula inductively, the proposed weighting technique realizes a good direction of update. The resulting weights turn out to be pairwise optimal in a certain sense. The proposed algorithm has the inherently parallel structure composed of q primitive functions, hence its total computational complexity O(qrN) is reduced to O(rN) with q concurrent processors (r: a constant positive integer). Numerical examples demonstrate that the proposed technique for r=1 yields significantly faster convergence than not only adaptive PSP with uniform weights, affine projection algorithm, and fast Newton transversal filters but also the regularized recursive least squares algorithm

Adaptive Parallel Quadratic-Metric Projection Algorithms

This paper indicates that an appropriate design of metric leads to significant improvements in the adaptive projected subgradient method (APSM), which unifies a wide range of projection-based algorithms [including normalized least mean square (NLMS) and affine projection algorithm (APA)]. The key is to incorporate a priori (or a posteriori) information on characteristics of an estimandum, a system to be estimated, into the metric design. We propose a family of efficient adaptive filtering algorithms based on a parallel use of quadratic-metric projection, which assigns every point to the nearest point in a closed convex set in a quadratic-metric sense. We present two versions: (1) constant-metric and (2) variable-metric, i.e., the metric function employed is (1) constant and (2) variable among iterations. As a constant-metric version, adaptive parallel quadratic-metric projection (APQP) and adaptive parallel min-max quadratic-metric projection (APMQP) algorithms are naturally derived by APSM, being endowed with desirable properties such as convergence to a point optimal in asymptotic sense. As a variable-metric version, adaptive parallel variable-metric projection (APVP) algorithm is derived by a generalized APSM, enjoying an extended monotone property at each iteration. By employing a simple quadratic-metric, the computational complexity of the proposed algorithms is kept linear with respect to the filter length. Numerical examples demonstrate the remarkable advantages of the proposed algorithms in an application to acoustic echo cancellation.

Krylov-Proportionate Adaptive Filtering Techniques Not Limited to Sparse Systems

This paper proposes a novel adaptive filtering scheme named the Krylov-proportionate normalized least-mean-square (KPNLMS) algorithm. KPNLMS exploits the benefits (i.e., fast convergence for sparse unknown systems) of the proportionate NLMS algorithm, but its applications are not limited to sparse unknown systems. A set of orthonormal basis vectors is generated from a certain Krylov sequence. It is proven that the unknown system is sparse with respect to the basis vectors in case of fairly uncorrelated input data. Different adaptation gain is allocated to a coefficient of each basis vector, and the gain is roughly proportional to the absolute value of the corresponding coefficient of the current estimate. KPNLMS enjoys i) fast convergence, ii) linear complexity per iteration, and iii) no use of any a priori information. Numerical examples demonstrate significant advantages of the proposed scheme over the reduced-rank method based on the multistage Wiener filter (MWF) and the transform-domain adaptive filter (TDAF) both in noisy and silent situations.

Efficient Acoustic Echo Cancellation With Reduced-Rank Adaptive Filtering Based on Selective Decimation and Adaptive Interpolation

This paper presents a new approach to efficient acoustic echo cancellation (AEC) based on reduced-rank adaptive filtering equipped with selective-decimation and adaptive interpolation. We propose a novel structure of an AEC scheme that jointly optimizes an interpolation filter, a decimation unit, and a reduced-rank filter. With a practical choice of parameters in AEC, the total computational complexity of the proposed reduced-rank scheme with the normalized least mean square (NLMS) algorithm is approximately half of that of the full-rank NLMS algorithm. We discuss the convergence properties of the proposed scheme and present a convergence condition. First, we examine the performance of the proposed scheme in a single-talk situation with an error-minimization criterion adopted in the decimation selection. Second, we investigate the potential of the proposed scheme in a double-talk situation by employing an ideal decimation selection. In addition to mean squared error (MSE) and power spectrum analysis of the echo estimation error, subjective assessments based on absolute category rating are performed, and the results demonstrate that the proposed structure provides significant improvements compared to the full-rank NLMS algorithm.

A unified view of adaptive variable-metric projection algorithms

We present a unified analytic tool named variable-metric adaptive projected subgradient method (V-APSM) that encompasses the important family of adaptive variable-metric projection algorithms. The family includes the transform-domain adaptive filter, the Newton-method-based adaptive filters such as quasi-Newton, the proportionate adaptive filter, and the Krylov-proportionate adaptive filter. We provide a rigorous analysis of V-APSM regarding several invaluable properties including monotone approximation, which indicates stable tracking capability, and convergence to an asymptotically optimal point. Small metric-fluctuations are the key assumption for the analysis. Numerical examples show (i) the robustness of V-APSM against violation of the assumption and (ii) the remarkable advantages over its constant-metric counterpart for colored and nonstationary inputs under noisy situations.

Coordinated Beamforming With Relaxed Zero Forcing: The Sequential Orthogonal Projection Combining Method and Rate Control

In this paper, coordinated beamforming based on relaxed zero forcing (RZF) for transmitter-receiver pair multiple-input single-output (MISO) and multiple-input multiple-output (MIMO) interference channels is considered. In the RZF coordinated beamforming, conventional zero-forcing interference leakage constraints are relaxed so that some predetermined interference leakage to undesired receivers is allowed in order to increase the beam design space for larger rates than those of the zero-forcing (ZF) scheme or to make beam design feasible when ZF is impossible. In the MISO case, it is shown that the rate-maximizing beam vector under the RZF framework for a given set of interference leakage levels can be obtained by sequential orthogonal projection combining (SOPC). Based on this, exact and approximate closed-form solutions are provided in two-user and three-user cases, respectively, and an efficient beam design algorithm for RZF coordinated beamforming is provided in general cases. Furthermore, the rate control problem under the RZF framework is considered. A centralized approach and a distributed heuristic approach are proposed to control the position of the designed rate-tuple in the achievable rate region. Finally, the RZF framework is extended to MIMO interference channels by deriving a new lower bound on the rate of each user.