Meng Zhang

Sparse RBF Networks with Multi-kernels

While the conventional standard radial basis function (RBF) networks are based on a single kernel, in practice, it is often desirable to base the networks on combinations of multiple kernels. In this paper, a multi-kernel function is introduced by combining several kernel functions linearly. A novel RBF network with the multi-kernel is constructed to obtain a parsimonious and flexible regression model. The unknown centres of the multi-kernels are determined by an improved k-means clustering algorithm. And orthogonal least squares (OLS) algorithm is used to determine the remaining parameters. The complexity of the newly proposed algorithm is also analyzed. It is demonstrated that the new network can lead to a more parsimonious model with much better generalization property compared with the traditional RBF networks with a single kernel.

Harmonic retrieval in complex noises based on wavelet transform

A novel approach for frequency estimation of one-dimensional harmonics in multiplicative and additive noises is presented. To overcome the resolution limitation inherent to the traditional Fourier-based algorithms, a wavelet transform is utilized. In this new approach, we use a wavelet mother function with a tunable parameter, which is constructed by modulating a window function. For a given harmonic retrieval problem, the tunable parameter can be adaptively adjusted to achieve a good performance. Some numerical experiments are included to illustrate the merits of this new approach.

Hybrid wavelet model construction using orthogonal forward selection with boosting search

This paper considers sparse regression modelling using a generalised kernel model in which each kernel regressor has its individually tuned centre vector and diagonal covariance matrix. An Orthogonal Least Squares (OLS) forward selection procedure is employed to select the regressors one by one using a guided random search algorithm. In order to prevent the possible overfitting, a practical method to select the termination threshold is used. A novel hybrid wavelet is constructed to make the model sparser. The experimental results show that this generalised model outperforms the traditional methods in terms of precision and sparseness. The model with the wavelet and hybrid kernel has a much faster convergence rate compared to that with a conventional Radial Basis Function (RBF) kernel.

Improved Orthogonal Least-Squares Regression With Tunable Kernels Using a Tree Structure Search Algorithm

Orthogonal least-squares (OLS) regression with tunable kernels has been recently introduced, in which a greedy scheme is utilized to tune the parameters of each individual regressor term by term using a global search algorithm. To improve the performance of the greedy-scheme-based OLS algorithm, a tree structure search algorithm is constructed. At each regressor stage, this proposed OLS algorithm is realized by keeping multiple best regressors rather than using the optimal one only. Numerical results show that this new scheme is capable of producing a much sparser regression model with better generalization than the conventional approaches.

Hybrid Wavelet Model Construction Using Orthogonal Forward Selection with Boosting Search

This paper considers sparse regression modeling using a generalized kernel model in which each kernel regressor has its individually tuned center vector and diagonal covariance matrix. An orthogonal least squares forward selection procedure is employed to select the regressors one by one using a guided random search algorithm. In order to prevent the possible over-fitting, a practical method to select termination threshold is used. A novel hybrid wavelet is constructed to make the model sparser. The experimental results show that this generalized model outperforms traditional methods in terms of precision and sparseness. And the models with wavelet and hybrid kernel have a much faster convergence rate as compared to that with conventional RBF kernel.

Robust Frequency Estimation of Multi-sinusoidal Signals Using Orthogonal Matching Pursuit with Weak Derivatives Criterion

In this paper, the weak derivatives (WD) criterion is introduced to solve the frequency estimation problem of multi-sinusoidal signals corrupted by noises. The problem is therefore modeled as a new least squares optimization task combined with WD. To overcome the potential basis mismatch effect caused by discretization of the frequency parameters, a modified orthogonal matching pursuit algorithm is proposed to solve the optimization problem by coupling it with a novel multi-grid dictionary training strategy. The proposed algorithm is validated on a set of simulated datasets with white noise and stationary colored noise. The comprehensive simulation studies show that the proposed algorithm can achieve more accurate and robust estimation than state-of-the-art algorithms.

Non-flat Function Estimation Using Orthogonal Least Squares Regression with Multi-scale Wavelet Kernel

Estimating the non-flat function which comprises both the steep variations and the smooth variations is a hard problem. The existing kernel methods with a single common variance for all the regressors can not achieve satisfying results. In this paper, a novel multi-scale model is constructed to tackle the problem by orthogonal least squares regression (OLSR) with wavelet kernel. The scheme tunes the dilation and translation of each wavelet kernel regressor by incrementally minimizing the training mean square error using a guided random search algorithm. In order to prevent the possible over-fitting, a practical method to select termination threshold is used. The experimental results show that, for non-flat function estimation problem, OLSR outperforms traditional methods in terms of precision and sparseness. And OLSR with wavelet kernel has a faster convergence rate as compared to that with conventional Gaussian kernel.

Harmonic Retrieval Based on LS-SVM with Wavelet Kernel

In this paper, a novel harmonic retrieval scheme is proposed, which is based on least squares support vector machines (LS-SVM). We consider harmonic signals corrupted by additive noises. The harmonic model is expended by wavelet series, and the corresponding parameters are estimated by weighted LS-SVM approach. Wavelet kernel is adopted to enhance the resolution. Simulations performed on synthetic signals show some merits of the proposed method