Lee K. Jones

Constructive approximations for neural networks by sigmoidal functions

A constructive algorithm for uniformly approximating real continuous mappings by linear combinations of bounded sigmoidal functions is given. G. Cybenko (1989) has demonstrated the existence of uniform approximations to any continuous f provided that sigma is continuous; the proof is nonconstructive, relying on the Hahn-Branch theorem and the dual characterization of C(I/sup n/). Cybenkos result is extended to include any bounded sigmoidal (even nonmeasurable ones). The approximating functions are explicitly constructed. The number of terms in the linear combination is minimal for first-order terms. >

General entropy criteria for inverse problems, with applications to data compression, pattern classification, and cluster analysis

Minimum distance approaches are considered for the reconstruction of a real function from finitely many linear functional values. An optimal class of distances satisfying an orthogonality condition analogous to that enjoyed by linear projections in Hilbert space is derived. These optimal distances are related to measures of distances between probability distributions recently introduced by C.R. Rao and T.K. Nayak (1985) and possess the geometric properties of cross entropy useful in speech and image compression, pattern classification, and cluster analysis. Several examples from spectrum estimation and image processing are discussed. >

Periodic signals for measuring nonlinear Volterra kernels

The frequency-domain measurement of the Volterra kernels of a nonlinear system using periodic multisine signals is now a practical possibility. An analysis is presented of the harmonic output of a Volterra kernel when excited with a multiharmonic signal, which lays the basis for the design of such signals. This is followed by a review of previous work in this area, after which a range of new periodic signals is defined. The minimization of the signal crest factors is then examined, along with the practical problems associated with their application. Practical results are presented which illustrate the application of the signals to testing a reference nonlinear circuit and a servo motor system.

Nonlinear disturbance errors in system identification using multisine test signals

The errors introduced into linear system identification by nonlinear distortions are examined. A theoretical framework is presented for the distortion generated by odd power nonlinearities when using multisine test signals for frequency domain identification. It is shown that the distortion is a function of the number of test harmonics, their harmonic values and their phases. An explanation of previously published practical results is then given. This leads to the definition of a novel class of signals, termed no interharmonic distortion (NID) multisines, with interesting properties. The application of NID multisines to system testing with a recently proposed method of compensating for nonlinearities is examined. This allows the identification of the linear system and the straightforward calculation of the coefficient of the nonlinear term. >

Approximation-theoretic derivation of logarithmic entropy principles for inverse problems and unique extension of the maximum entropy method to incorporate prior knowledge

The minimum distance approach for reconstructing a positive function based on knowledge of finitely many linear functional values is examined. Two important classes of directed distances for signal processing and statistical inference are discussed. By imposing conditions analogous to those satisfied by linear projections in Hilbert space, two logarithmic entropy principles are derived. One of these involves the Itakura–Saito distortion measure of communication theory and uniquely extends Burgs maximum entropy method to incorporate prior knowledge. The other uses the Kullback–Leibler distance of statistics.

The computational intractability of training sigmoidal neural networks

We demonstrate that the problem of approximately interpolating a target function by a neural network is computationally intractable. In particular the interpolation training problem for a neural network with two monotone Lipschitzian sigmoidal internal activation functions and one linear output node is shown to be NP-hard and NP-complete if the internal nodes are in addition piecewise ratios of polynomials. This partially answers a question of Blum and Rivest (1992) concerning the NP-completeness of training a logistic sigmoidal 3-node network. An extension of the result is then given for networks with n monotone sigmoidal internal nodes and one convex output node. This indicates that many multivariate nonlinear regression problems may be computationally infeasible.

Pattern classification using projection pursuit

Abstract This article discusses the adaptation of recently developed regression techniques to classifier design. Apart from finite sample effects, projection pursuit (PP) regression may be used to model a desired response (class) as a sum of ridge functions according to a minimum expected squared error criterion. This approach can be shown to furnish an optimal discriminant function which can satisfy the Neyman-Pearson criterion over all possible thresholds. Basis function expansions are used instead of smoothed histograms to reduce computation. Since good approximation of a discriminant by a linear combination of moderate number of ridge functions may not be easy, we introduce an improved method utilizing a nonlinear weighting function.

Inferring Balking Behavior From Transactional Data

Balking is the act of not joining a queue because the prospective arriving customer judges the queue to be too long. We analyze queues in the presence of balking, using only the service start and stop data utilized in Larsons Queue Inference Engine (Q.I.E.). Using an extension of Larsons congestion probability calculation to include balking we present new maximum likelihood, nonparametric, and Bayesian methods for inferring the arrival rate and balking functions. The methodology is applicable to businesses that wish to estimate lost sales because of balking arising from queuing-type congestion. The techniques are applied to a small transactional data set for illustrative purposes.

On transformations without finite invariant measure

We show that any invertible nonsingular transformation T of a finite measure space ( Ω , Ol , μ ) admits a countable partition of Ω into disjoint measurable sets Ω 0 , Ω 1 , Ω 2 ,… so that (a) Ω 0 and ∪ i ⩾1 Ω i are invariant under T , (b) T restricted to Ω 0 has a finite equivalent invariant measure, (c) each Ω i is an image under an integral power of T of each Ω i ( i, j ⩾ 1). If Ol is countably generated mod μ the sets Ω i ( i ⩾ 1) can be constructed with the additional property of being strongly generating in Ω / Ω 0 . We also give a streamlined introduction to some known results on existence of invariant measures and, thereby, make the paper completely self-contained.

Unimodular eigenvalues and weak mixing

Abstract Let T be a linear operator on a Banach space X , with sup ∥ T n ∥ T n x } is weak- ∗ sequentially compact in X ∗∗ , we prove the equivalence of the following: (1) lim N → ∞ N −1 ∑ n = 1 N ¦〈 x ∗ , T n x〉¦ = 0 for every x ∗ ϵ X ∗ . (2) x is orthogonal to the eigenvectors of T ∗ with unimodular eigenvalues. For example, the assumption holds if X ∗ is separable. An example shows that the sufficient condition for (1), T n inix → 0 weakly for some { n i }, is not necessary.