Neil J. Bershad

Analysis of the normalized LMS algorithm with Gaussian inputs

The LMS adaptive filter algorithm requires a priori knowledge of the input power level to select the algorithm gain parameter μ for stability and convergence. Since the input power level is usually one of the statistical unknowns, it is normally estimated from the data prior to beginning the adaptation process. It is then assumed that the estimate is perfect in any subsequent analysis of the LMS algorithm behavior. In this paper, the effects of the power level estimate are incorporated in a data dependent μ that appears explicitly within the algorithm. The transient mean and second-moment behavior of the modified LMS (NLMS) algorithm are evaluated, taking into account the explicit statistical dependence of μ upon the input data. The mean behavior of the algorithm is shown to converge to the Wiener weight. A constant coefficient matrix difference equation is derived for the weight fluctuations about the Wiener weight. The equation is solved for a white data covariance matrix and for the adaptive line enhancer with a single-frequency input in steady state for small μ. Expressions for the misadjustment error are also presented. It is shown for the white data covariance matrix case that the averaging of about ten data samples causes negligible degradation as compared to the LMS algorithm. In the ALE application, the steady-state weight fluctuations are shown to be mode dependent, being largest at the frequency of the input.

Time delay estimation using the LMS adaptive filter--Dynamic behavior

A new application of the LMS adaptive filter, that of determining the time delay in a signal between two split-array outputs, is described. In a split array sonar, this time delay can be converted to the bearing of the target radiating the signal. The performance of such a tracker is analyzed for stationary broad-band targets. It is shown that a continuous adaptive tracker performs within 0.5 dB of the Cramer-Rao lower bound. Further, performance predictions are developed for a discrete adaptive tracker which demonstrates excellent agreement with simulations. It is shown that the adaptive tracker can have significantly less sensitivity to changing input spectra than a conventional tracker using a fixed input filter.

Adaptive recovery of a chirped sinusoid in noise. II. Performance of the LMS algorithm

The authors study the ability of the exponentially weighted recursive least square (RLS) algorithm to track a complex chirped exponential signal buried in additive white Gaussian noise (power P/sub n/). The signal is a sinusoid whose frequency is drifting at a constant rate Psi . lt is recovered using an M-tap adaptive predictor. Five principal aspects of the study are presented: the methodology of the analysis; proof of the quasi-deterministic nature of the data-covariance estimate R(k); a new analysis of RLS for an inverse system modeling problem; a new analysis of RLS for a deterministic time-varying model for the optimum filter; and an evaluation of the residual output mean-square error (MSE) resulting from the nonoptimality of the adaptive predictor (the misadjustment) in terms of the forgetting rate ( beta ) of the RLS algorithm. It is shown that the misadjustment is dominated by a lag term of order beta /sup -2/ and a noise term of order beta . Thus, a value beta /sub opt/ exists which yields a minimum misadjustment. It is proved that beta /sub opt/=((M+1) rho Psi /sup 2/)/sup 1/3/, and the minimum misadjustment is equal to (3/4)P/sub n/(M+1) beta /sub opt/, where rho is the input signal-to-noise ratio (SNR). >

An Affine Combination of Two LMS Adaptive Filters—Transient Mean-Square Analysis

This paper studies the statistical behavior of an affine combination of the outputs of two least mean-square (LMS) adaptive filters that simultaneously adapt using the same white Gaussian inputs. The purpose of the combination is to obtain an LMS adaptive filter with fast convergence and small steady-state mean-square deviation (MSD). The linear combination studied is a generalization of the convex combination, in which the combination factor lambda(n) is restricted to the interval (0,1). The viewpoint is taken that each of the two filters produces dependent estimates of the unknown channel. Thus, there exists a sequence of optimal affine combining coefficients which minimizes the mean-square error (MSE). First, the optimal unrealizable affine combiner is studied and provides the best possible performance for this class. Then two new schemes are proposed for practical applications. The mean-square performances are analyzed and validated by Monte Carlo simulations. With proper design, the two practical schemes yield an overall MSD that is usually less than the MSDs of either filter.

A frequency domain model for 'filtered' LMS algorithms-stability analysis, design, and elimination of the training mode

A frequency domain model of the filtered LMS algorithm is presented for analyzing the behavior of the weights during adaptation. In particular, expressions for stable operation of the algorithm are derived as a function of the algorithm step size, the input signal power, and the transfer functions of the linear filters. The expressions show that algorithm stability can be achieved over a frequency band of interest by inserting an appropriately chosen delay in the reference input to the LMS algorithm weight update equation. This result implies that it is not necessary to use a training mode to estimate the loop transfer functions before or during adaptation if the input is limited to a band of frequencies. It is only necessary to know the approximate delay introduced by the transfer functions in the band. The single delay parameter can be estimated much more easily than the entire transfer function. Simulations of the time domain algorithm are presented to support the theoretical predictions of the frequency domain model. >

Mean weight behavior of the filtered-X LMS algorithm

A new stochastic analysis is presented for the filtered-X LMS (FXLMS) algorithm. The analysis does not use independence theory. An analytical model is derived for the mean behavior of the adaptive weights. The model is valid for white or colored reference inputs and accurately predicts the mean weight behavior even for large step sizes. The constrained Wiener solution is determined as a function of the input statistics and the impulse responses of the adaptation loop filters. Effects of secondary path estimation error are studied. Monte Carlo simulations demonstrate the accuracy of the theoretical model.

On error-saturation nonlinearities in LMS adaptation

The effect of a saturation-type error nonlinearity in the weight update equation in least-mean-squares (LMS) adaptation is investigated for a white Gaussian data model. Nonlinear difference equations are derived for the eight first and second moments, which include the effect of an error function (erf) saturation-type nonlinearity on the error sequence driving the algorithm. A nonlinear difference equation for the mean norm is explicitly solved using a differential equation approximation and integration by quadratures. The steady-state second-moment weight behavior is evaluated exactly for the erf nonlinearity. Using the above results, the tradeoff between the extent of error saturation, steady-state excess mean-square error, and rate of algorithm convergence is studied. The tradeoff shows that (1) starting with a sign detector, the convergence rate is increased by nearly a factor of two for each additional bit, and (2) as the number of bits is increased further, the additional bit by very little in convergence speed, asymptotically approaching the behavior of the linear algorithm. >

Neural network modeling and identification of nonlinear channels with memory: algorithms, applications, and analytic models

This paper proposes a neural network (NN) approach for modeling nonlinear channels with memory. Two main examples are given: (1) modeling digital satellite channels and (2) modeling solid-state power amplifiers (SSPAs). NN models provide good generalization performance (in terms of output signal-to-error ratio). NN modeling of digital satellite channels allows the characterization of each channel component. Neural net models represent the SSPA as a system composed of a linear complex filter followed by a nonlinear memoryless neural net followed by a linear complex filter. If the new algorithms are to be used in real systems, it is important that the algorithm designer understands their learning behavior and performance capabilities. Some simplified neural net models are analyzed in support of the simulation results. The analysis provides some theoretical basis for the usefulness of NNs for modeling satellite channels and amplifiers. The analysis of the simplified adaptive models explains the simulation results qualitatively but not quantitatively. The analysis proceeds in several steps and involves several novel ideas to avoid solving the more difficult general nonlinear problem.

Tracking characteristics of the LMS adaptive line enhancer-response to a linear chirp signal in noise

The transient behavior of the LMS adaptive filter is studied when configured as an adaptive line enhancer operating in the presence of a fixed or variable complex frequency sine-wave signal buried in white noise. For a fixed frequency signal, the mean weights are shown to respond to signal more rapidly than to noise alone. For a chirped signal, a fixed parameter matrix first-order difference equation is derived for the mean weights and a closed-form steady-state solution obtained. The transient response is obtained as a function of the eigenvectors and eigenvalues of the input covariance matrix. Sufficient conditions for the stability of the transient response are derived and an upper bound on the eigenvalues obtained. Finally, the mean-square error is evaluated when responding to a chirped signal. A gain coefficient of the LMS algorithm is determined which minimizes the mean-square error for chirped signals as a function of chirp rate and signal and noise powers.

A statistical analysis of the affine projection algorithm for unity step size and autoregressive inputs

This paper presents a new statistical analysis of the affine projection (AP) algorithm. An analytical model is derived for autoregressive (AR) inputs for unity step size (fastest convergence). Deterministic recursive equations are derived for the mean AP weight and mean-square error for large values of N (the number of adaptive taps). The value of N is also assumed large compared to the algorithm order (number of input vectors used to determine the weight update direction). The model predictions display better agreement with Monte Carlo simulations in both transient and steady-state than models previously presented in the literature. The models accuracy is sufficient for most practical design purposes.