Masao Yamagishi

A sparse adaptive filtering using time-varying soft-thresholding techniques

In this paper, we propose a novel adaptive filtering algorithm based on an iterative use of (i) the proximity operator and (ii) the parallel variable-metric projection. Our time-varying cost function is a weighted sum of squared distances (in a variable-metric sense) plus a possibly nonsmooth penalty term, and the proposed algorithm is derived along the idea of proximal forward-backward splitting in convex analysis. For application to sparse-system identification problems, we employ the (weighted) ℓ1 norm as the penalty term, leading to a time-varying soft-thresholding operator. As the simple example of the proposed algorithm, we present the variable-metric affine projection algorithm composed with the time-varying soft-thresholding operator. Numerical examples demonstrate that the proposed algorithms notably outperform their counterparts without soft-thresholding both in convergence speed and steady-state mismatch, while the extra computational complexity due to the additional soft-thresholding is negligibly low.

Minimizing the moreau envelope of nonsmooth convex functions over the fixed point set of certain quasi-nonexpansive mappings

The first aim of this paper is to present a useful toolbox of quasi-nonexpansive mappings for convex optimization from the viewpoint of using their fixed point sets as constraints. Many convex optimization problems have been solved through elegant translations into fixed point problems. The underlying principle is to operate a certain quasi-nonexpansive mapping T iteratively and generate a convergent sequence to its fixed point. However, such a mapping often has infinitely many fixed points, meaning that a selection from the fixed point set Fix(T) should be of great importance. Nevertheless, most fixed point methods can only return an “unspecified” point from the fixed point set, which requires many iterations. Therefore, based on common sense, it seems unrealistic to wish for an “optimal” one from the fixed point set. Fortunately, considering the collection of quasi-nonexpansive mappings as a toolbox, we can accomplish this challenging mission simply by the hybrid steepest descent method, provided that the cost function is smooth and its derivative is Lipschitz continuous. A question arises: how can we deal with “nonsmooth” cost functions? The second aim is to propose a nontrivial integration of the ideas of the hybrid steepest descent method and the Moreau–Yosida regularization, yielding a useful approach to the challenging problem of nonsmooth convex optimization over Fix(T). The key is the use of smoothing of the original nonsmooth cost function by its Moreau–Yosida regularization whose the derivative is always Lipschitz continuous. The field of application of hybrid steepest descent method can be extended to the minimization of the ideal smooth approximation Fix(T). We present the mathematical ideas of the proposed approach together with its application to a combinatorial optimization problem: the minimal antenna-subset selection problem under a highly nonlinear capacity-constraint for efficient multiple input multiple output (MIMO) communication systems.

Active Data Selection for Motor Imagery EEG Classification

Rejecting or selecting data from multiple trials of electroencephalography (EEG) recordings is crucial. We propose a sparsity-aware method to data selection from a set of multiple EEG recordings during motor-imagery tasks, aiming at brain machine interfaces (BMIs). Instead of empirical averaging over sample covariance matrices for multiple trials including low-quality data, which can lead to poor performance in BMI classification, we introduce weighted averaging with weight coefficients that can reject such trials. The weight coefficients are determined by the ℓ1-minimization problem that lead to sparse weights such that almost zero-values are allocated to low-quality trials. The proposed method was successfully applied for estimating covariance matrices for the so-called common spatial pattern (CSP) method, which is widely used for feature extraction from EEG in the two-class classification. Classification of EEG signals during motor imagery was examined to support the proposed method. It should be noted that the proposed data selection method can be applied to a number of variants of the original CSP method.

A sparse system identification by using adaptively-weighted total variation via a primal-dual splitting approach

Observing that sparse systems are almost smooth, we propose to utilize the newly-introduced adaptively-weighted total variation (AWTV) for sparse system identification. In our formulation, a sparse system identification problem is posed as a sequential suppression of a time-varying cost function: the sum of AWTV and a data-fidelity term. In order to handle such a non-differentiable cost function efficiently, we propose a time-varying extension of a primal-dual splitting type algorithm, named the adaptive primal-dual splitting method (APDS). APDS is free from operator inversion or other highly complex operations, resulting in computationally efficient implementation in online manner. Moreover, APDS realizes that the sequence defined in a certain product space monotonically approaches the solution set of the current cost function, i.e., the sequence generated by APDS pursues desired replicas of the unknown system in each time-step. Our scheme is applied to a network echo cancellation problem where it shows excellent performance compared with conventional methods.

Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize

In this paper, we present an over-relaxed variant of the fast iterative shrinkage-thresholding algorithm (FISTA)/the monotone FISTA (MFISTA). FISTA and MFISTA are iterative first-order algorithms, whose convergence rates of the objective function are for an iteration counter k, for the minimization of the sum of a smooth and a nonsmooth convex function. FISTA and MFISTA are composed of the forward–backward splitting step together with a certain computationally efficient shifting step. The stepsize available in the forward–backward splitting step in these algorithms has been limited to a fixed value determined by the Lipschitz constant of the gradient of the smooth function. Examples of the proposed scheme admit variable stepsizes in broader ranges than FISTA/MFISTA, while keeping the same convergence rate . A numerical example in a well-conditioned case demonstrates the effect of the proposed relaxations by showing that the proposed scheme outperforms, in the speed of convergence, the original FISTA and MFISTA.

Acceleration of adaptive proximal forward-backward splitting method and its application to sparse system identification

In this paper, we propose an acceleration technique of the adaptive filtering scheme called adaptive proximal forward-backward splitting method. For accelerating the convergence rate, the proposed method includes a step to shift the current estimate in the direction of the difference between the current and previous estimates based on the Fast Iterative Shrinkage/Thresholding Algorithm (FISTA). The computational complexity for this additional step is fairly low compared to the overall complexity of the algorithm. As an example of the proposed method, we derive an acceleration of the composition of the Adaptively Weighted Soft-Thresholding (AWST) operator and the exponentially weighted adaptive parallel projection. AWST shrinks the estimated filter coefficients to zero for exploiting the sparsity of the system to be estimated and the exponentially weighted adaptive parallel projection algorithm realizes high accuracy by utilizing all available information at each iteration. This accelerated method improves the steady-state mismatch drastically with its convergence speed as fast as the proportionate affine projection algorithm.

Graph Signal Denoising via Trilateral Filter on Graph Spectral Domain

This paper presents a graph signal denoising method with the trilateral filter defined in the graph spectral domain. The original trilateral filter (TF) is a data-dependent filter that is widely used as an edge-preserving smoothing method for image processing. However, because of the data-dependency, one cannot provide its frequency domain representation. To overcome this problem, we establish the graph spectral domain representation of the data-dependent filter, i.e., a spectral graph TF (SGTF). This representation enables us to design an effective graph signal denoising filter with a Tikhonov regularization. Moreover, for the proposed graph denoising filter, we provide a parameter optimization technique to search for a regularization parameter that approximately minimizes the mean squared error w.r.t. the unknown graph signal of interest. Comprehensive experimental results validate our graph signal processing-based approach for images and graph signals.

Sparsity-aware adaptive filters based on ℓ p -norm inspired soft-thresholding technique

We propose a novel sparsity-aware adaptive filtering algorithm based on iterative use of weighted soft-thresholding. The weights are determined based on a rough local approximation of the ℓp norm (0 <; p <; 1). The proposed algorithm operates the weighted soft-thresholding for enhancing the sparsity, following estimation error managements with the affine projection. The proposed weighting technique alleviates an extra bias of no benefit caused by shrinking dominant coefficients. The numerical examples demonstrate that the proposed weighting technique outperforms the existing one when the situation changes under the fixed parameter settings.

A rank selection of MV-PURE with an unbiased predicted-MSE criterion and its efficient implementation in image restoration

The Minimum-Variance Pseudo-Unbiased Reduced-rank Estimator (MV-PURE) is designed, as a natural reduced-rank extension of the Gauss-Markov estimator, for the unknown deterministic vector in ill-conditioned linear regression model. In this paper, we propose a novel rank-selection for the MV-PURE to achieve a small Mean Square Error (MSE). The proposed rank-selection is realized by minimizing an unbiased estimate of the predicted-MSE, not of the MSE. Our unbiased estimate can be applicable to any noise distribution with zero mean and a finite covariance matrix, while Stein-type unbiased criteria cannot in general. We apply the proposed selection to an image restoration problem and introduce its efficient O(m log m) implementation by using a special structure found in typical blur matrices, where the blur matrix is of size m×m. A numerical example demonstrates that the MV-PURE with the proposed rank-selection achieves a MSE comparable with the minimal MSE for the unknown vector among all possible ranks.

A Deep Monotone Approximation Operator Based on the Best Quadratic Lower Bound of Convex Functions

This paper presents a closed form solution to a problem of constructing the best lower bound of a convex function under certain conditions. The function is assumed (I) bounded below by -ρ, and (II) differentiable and its derivative is Lipschitz continuous with Lipschitz constant L. To construct the lower bound, it is also assumed that we can use the values ρ and L together with the values of the function and its derivative at one specified point. By using the proposed lower bound, we derive a computationally efficient deep monotone approximation operator to the level set of the function. This operator realizes better approximation than subgradient projection which has been utilized, as a monotone to level sets of differentiable convex functions as well as nonsmooth convex functions. Therefore, by using the proposed operator, we can improve many signal processing algorithms essentially based on the subgradient projection.